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I.   INTRODUCTION 

This  study  is  a  comparison  of  two  state-of-the-art  nonlinear  program- 
ming codes  that  are  designed  to  accommodate  problems  of  thousands  of 
variables  and  constraints.  While  there  are  many  codes  designed  to  handle 
the  general  linear  programming  (LP)  problem  and  its  specializations, 
there  are  significantly  fewer  systems  designed  to  reliably  solve  the  much 
more  difficult  nonlinear  programming  (NLP)  problem.  Of  these,  \iery   few 
are  capable  of  solving  "large-scale"  problems:  larger  than,  say,  a 
thousand  constraints  or  a  thousand  variables  or  more.  Most  of  these 
large-scale  codes  are  internal,  proprietary  systems  developed  by  companies 
for  the  solution  of  their  specific  industrially  related  problems;  the 
codes  used  by  petroleum  refiners  for  chemical  process  control  provide 
singular  examples  of  such  contributions. 

One  of  the  codes  evaluated  in  this  study  is  MINOS  (j^odular  _In-core 
Nonlinear  Optimization  System)  developed  by  B.  A.  Murtagh,  the  University 
of  New  South  Wales,  and  M.  A.  Saunders,  Stanford  University.  The  other 
is  XS  (X  System)  developed  by  G.  G.  Brown,  Naval  Postgraduate  School,  and 
G.  W.  Graves,  University  of  California  at  Los  Angeles. 

This  is  the  first  independent  comparison  of  either  code  and  is 
intended  to  serve  both  as  an  evaluation  of  each  and  as  a  guide  to  the 
potential  user  concerned  with  the  applicability  of  each  code  to  the 
individual  problem  with  which  he  might  be  faced. 

Two  caveats  should  be  kept  in  mind  while  reading  this  evaluation. 
First,  the  codes  are  quite  different  in  intended  use.  MINOS  is  intended 
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as  an  academic  production  code  and  is  designed  to  be  readily  distributed 
and  applied  by  a  wide  variety  of  users.  Extensive  documentation  and 
reliable  performance  have  been  paramount  concerns  in  the  development  of 
MINOS.  On  the  other  hand,  XS  is  used  as  an  advanced  experimental  testbed 
for  optimization  research.  The  fully  instrumented  version  used  in  this 
comparison  is  a  prototype  designed  to  be  used  almost  exclusively  by  its 
originators  and  their  co-workers  for  a  wide  range  of  problems,  such  as 
large  mixed  integer  and  linear  formulations  and  especially  for  decomposi- 
tion problems.  As  such,  it  is  in  a  continual  state  of  flux  and  varies 
considerably  in  its  content  (hopefully  in  an  improving  direction)  from 
month  to  month.  All  results  from  XS  are  from  the  most  recent  prototype 
system  at  the  time  of  publication  cutoff  for  this  thesis  with  no  special- 
ization for  nonlinear  programming.  Academic  and  industrial  production 
versions  of  XS  are  typically  customized  to  the  application  at  hand  and 
thoroughly  documented  for  routine  use. 

Second,  although  both  systems  are  "large-scale"  nonlinear  codes  which 
have  been  successfully  used  on  many  large,  real-life  problems,  because  of 
their  intended  day-to-day  application,  their  characteristics  are  not  the 
same,  nor  are  they  intended  to  be.  Therefore,  any  differences  between 
them  in  speed  or  capability  may  be  attributable  to  design  intention  rather 
than  relative  deficiencies  in  the  algorithms,  underlying  data  structures, 
or  implementation. 

A.  GENERAL  PROBLEM  STATEMENT 

The  general  linear  programming  (LP)  problem  can  be  stated  as: 
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minimize    c  x  (objective  function) 

subject  to  r_  <_  A  x  <_  r     (ranged  constraints) 

b_  <_  x  <_  6     (bounds  on  variables) 
where: 

x    =  variables; 

c    =  cost  coefficients; 

A    =  constraint  matrix  coefficients; 

r_,  r  =  upper  and  lower  constraint  ranges; 

lb_,  6  =  upper  and  lower  variable  bounds. 
The  general  non-separable  nonlinear  (NLP)  problem  can  be  stated  as: 
minimize        f(x)       (objective  function) 
subject  to    L.SgU)  £  r    (ranged  constraints) 
b_  <_  x  <_  6     (bounds  on  variables) 
where 

x     =  variables; 

f(x)  =  general  non-separable,  nonlinear  function; 

gr(x)  =  general  non-separable,  nonlinear  constraint; 

r_,  r     =   upper  and  lower  constraint  ranges; 

b_,  b  =  upper  and  lower  variable  bounds. 

B.   COMPARISON  CRITERIA 

In  any  study  of  this  nature,  one  of  the  primary  concerns  is  the 
criteria  with  which  the  codes  are  to  be  objectively  compared.   In  this 
case,  the  guidelines  recently  published  in  Operations  Research  [Ref .  1] 
will  be  used  with  some  modification  to  prevent  comparisons  that  are  not 
valid  because  of  the  somewhat  different  nature  of  the  two  codes.  These 
criteria,  to  be  elaborated  in  Chapter  IV,  are  listed  below. 
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1.  Algorithm  Capabilities 

This  section  contains  a  general  overview  of  the  types  and  classes 
of  problems  for  which  each  code  is  designed  and  comments  on  the  growth 
capabilities  of  each  code. 

2.  CPU  (Compute)  Time 

The  CPU  times  listed  are  the  virtual  CPU  times  required  for  each 
problem  running  with  precompiled  load  modules  for  each  primary  system 
code  and  do  not  include  the  linkage  editor  times.  Since  the  problems 
have  been  run  interactively  on  a  virtual  memory  computer  system,  these 
times  will  vary  somewhat  from  run  to  run  depending  upon  computer  loading. 
Extensive  experience  on  the  host  computer  with  these  problems  indicates 
that  the  listed  CPU  times  are  valid  within  one  per  cent.  Although  the 
actual  "clock,"  or  "response,"  time  (as  opposed  to  CPU  time)  varies  as  a 
function  of  system  loading;  empirical  evidence  gathered  while  conducting 
this  study  suggests  that  a  useful  rule-of-thumb  is  that  actual  clock  time 
is  approximately  four  times  as  long  as  CPU  time  for  the  virtual  memory 
time-sharing  system  used.  This  should  provide  a  reasonable  estimate  of 
the  response  times  to  be  expected  for  problems  of  this  study. 

Because  of  the  region  requirements  of  the  FORTRAN  compiler  used 
on  the  host  computer  (see  Section  I.C),  one  megabyte  of  default  virtual 
memory  was  used  for  all  problems  in  a  single-step  procedure.  Comments 
concerning  problem-dependent  memory  requirements  of  each  system  will  be 
made  in  Chapter  IV. 

3.  Number  of  Iterations 

The  number  of  major  iterations  (linearizations)  and  pivots 
required  to  reach  solution  is  given  for  each  problem,  with  the  caveat 
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that  the  nature  of  an  "iteration"  varies  considerably  between  the  algo- 
rithms. The  specific  nature  of  these  iterations  is  discussed  in 
Sections  II. A. 1  and  II.B.l. 

4.  Number  of  Function  Evaluations 

The  number  of  function  evaluations  to  reach  solution  is  listed 
for  each  algorithm.  However,  since  this  number  includes  both  objective 
function  and  constraint  evaluations,  as  well  as  gradient  calculations  in 
the  case  of  MINOS,  different  amounts  of  information  may  be  obtained  on 
each  function  call  and  this  may,  therefore,  be  a  deceptive  comparison. 

5.  User  Friendliness 

One  of  the  primary  goals  of  this  study  is  to  evaluate  the  ability 
of  a  user  familiar  with  some  optimization  theory  but  with  little  experi- 
ence with  the  individual  codes  to  set  up  and  successfully  solve  a  problem, 
Because  of  the  codes'  different  design  motivations,  it  was  expected  from 
the  beginning  of  the  study  that  MINOS  would  be  far  superior  in  this 
regard. 

a.  Ease  of  Setup 

One  measure  of  the  flexibility  of  a  problem-solving  system 
is  the  ease  with  which  it  can  be  adapted  by  the  general  user  to  the 
particular  problem/data  structure  at  hand. 

b.  Debug  Output 

During  initial  debugging  of  a  problem,  varying  quantities 
and  types  of  diagnostic  information  may  be  required  to  isolate  a  par- 
ticular error.  The  ability  of  each  code  to  provide  a  tailored  output  in 
concise,  readable  form  for  the  user  will  be  evaluated. 
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c.  Failure  Mode 

Since  the  perfect  optimization  code  has  yet  to  be  developed, 
one  measure  of  a  code's  performance  is  its  ability  to  fail  "gracefully," 
leaving  the  user  in  a  posture  from  which  he  can  recover  without  all  of 
his  effort  being  wasted.  The  information  given  to  the  user  when  each 
code  fails  is  examined  to  evaluate  its  usefulness  in  further  problem 
exploration. 

d.  Robustness 

As  an  aid  to  the  inexperienced  user,  codes  should  be  robust 
in  their  default  parameters  to  minimize  the  amount  of  "tuning"  that  need 
be  done  on  most  problems.  At  the  same  time,  the  parameters  must  have 
sufficient  scope  and  power  to  allow  the  experienced  user  to  exploit  the 
structure  of  difficult  problems  where  interaction  by  the  analyst  is 
required  in  order  to  achieve  a  solution. 

C.   TEST  PROBLEMS 

This  study  has  been  handicapped,  as  have  other  similar  efforts,  by 
the  lack  of  suitable  test  problems  to  test  the  full  capabilities  of  the 
codes.   In  this  case,  the  need  has  been  for  large  (several  hundred 
variables  or  more)  real-life  problems.  The  author  has  been  unable  to 
secure  such  problems  for  which  publication  release  is  available  and  for 
which  the  data  is  in  a  form  that  is  readily  usable  by  both  codes.  This 
is  a  continuing  and  widely  recognized  problem  which  has  prompted  the 
development  of  a  number  of  nonlinear  artificial  problem  generators. 
However,  although  these  generators  can  produce  arbitrarily  large 
problems,  it  has  been  the  experience  of  the  developers  of  the  X  System 
that  the  randomly  generated  problems  produced  by  the  generators  are 
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not  realistic  tests  for  optimization  codes  because  they  possess  none  of 
the  specialized  structure  that  is  routinely  found  in  real-life  problems 
and  which,  in  fact,  is  the  \/ery  thing  capitalized  upon  by  good  codes  to 
produce  their  excellent  results  on  large  problems. 

Therefore,  for  the  purposes  of  this  test,  the  emphasis  has  been 
placed  on  real-life,  or  at  least  well-known  problems,  as  opposed  to 
generated  problems,  at  the  sacrifice  of  size.  Thirteen  problems  were 
selected  for  this  study,  of  which  at  least  nine  have  been  previously 
published.  The  variables  in  the  problems  range  from  two  to  793  and  the 
constraints  from  one  to  401.  The  problems  contain  a  mix  of  linear, 
nonlinear,  equality,  and  inequality  constraints.  Also  included  are  two 
problems  with  (12  and  100)  integer  variables  which  have  never  before  been 
formally  solved  as  nonlinear  integer  and  nonlinear  mixed  integer  programs 

D.   COMPUTER  SYSTEM 

All  computing  has  been  completed  in  the  W.  R.  Church  Computer  Center 
of  the  Naval  Postgraduate  School,  Monterey,  California  on  its  installed 
IBM  3033  computers  using  the  VM/SP  timesharing  system.  The  load  modules 
for  both  optimization  systems  and  for  each  respective  problem  generation 
subroutine  were  generated  with  the  FORTRAN  IV  (H  Extended)  compiler  using 
the  OPTIMIZE  (2)  option  [Ref.  2].  All  problems  have  been  solved  inter- 
actively in  real-time  on  the  computer  system  using  precompiled  load 
modules  of  the  respective  optimization  systems  linked  with  code  and 
parameter  data  for  the  individual  problems.  This  research  has  promoted 
development  of  an  extensive  real-time  library  of  service  routines  to  aid 
in  the  preparation,  execution,  and  interpretation  of  large  optimization 
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problems.  Although  neither  optimization  system  has  a  truly  interactive 
solution  algorithm,  each  can  be  used  with  interactive  parameter  settings 
and  with  real-time  monitoring  of  solution  progress,  providing  a  fertile 
research  environment. 
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II.   DESCRIPTION  OF  SYSTEMS 

A.   MINOS 

The  MINOS  version  discussed  in  this  study  is  MINOS/AUGMENTED  (alias 
MINOS  Version  4.0)  which  has  been  developed  as  an  extension  of  an  earlier 
MINOS  vintage  which  solved  problems  with  nonlinear  objective  functions, 
but  with  strictly  linear  constraints.  MINOS/AUGMENTED  (henceforth  called 
MINOS)  is  a  general -purpose  nonlinear  programming  system  designed  to 
solve  large-scale  optimization  problems  exhibiting  linear  and  nonlinear 
constraints,  linear  and  nonlinear  variables,  and  exploiting  sparsity 
(relatively  few  non-null  constraint-variable  interactions)  and  exclusive 
linearity  of  some  constraints  and  variables  (respectively  expressed  in  no 
nonlinear  terms).  Nonlinear  functions  in  a  problem  should  be  continuous 
with  continuous  first  derivatives,  but  need  not  be  separable.  Integer 
variables  are  not  accommodated.  The  user  specifies  nonlinear  objective 
functions  with  one  FORTRAN  subroutine,  nonlinear  constraint  functions  with 
a  second  FORTRAN  subroutine,  while  the  linear  portions  of  the  objective 
function  and  constraints,  ranges,  bounds,  and  initial  starting  point  (if 
any)  are  specified  in  standard  "MPS  Format"  [Ref.  3]. 

MINOS  employs  an  augmented  Lagrangian  algorithm  to  solve  problems 
with  nonlinear  constraints.  This  algorithm  uses  a  sequence  of  sparse, 
linearly  constrained  subproblems  which  are  solved  using  a  reduced-gradient 
algorithm. 

MINOS  is  intended  (as  cautioned  by  its  developers)  as  an  extension 
of  (not  a  replacement  for)  commercial  mathematical  programming  systems. 
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Consequently,  MINOS  does  not  possess  many  of  the  algorithmic  options 
(e.g.,  dual  simplex)  or  data  revision  and  file  handling  capabilities 
common  to  various  commercial  optimization  systems.  A  complete  descrip- 
tion of  the  code  can  be  found  in  the  support  documentation  [Ref.  4,  5,  6]. 
1.  Algorithm 

A  discussion  of  the  MINOS  algorithm  is  logically  divided  into 
two  cases: 

a.  Linear  Constraints 

Where  only  linear  constraints  are  present,  MINOS  is  designed 
to  solve  problems  of  the  form: 

minimize     f(x)  +  c  x 

subject  to    Ax  =  b 

b_<   x  <   b 

where  f(x)  is  a  continuous,  continuously  differentiate  function 

with  gradient: 

v  f(x)  =  (af/axj)  =  g(x). 
In  general,  the  constraint  matrix  A  is  assumed  to  be  large  and  sparse. 

The  foundation  of  MINOS  is  an  efficient  and  reliable  imple- 
mentation of  the  revised  simplex  method  for  LP  [Ref.  7].  This  combines 
sparse  matrix  technology  [Ref.  8:  pp.  213-226]  with  stable  numerical 
methods  for  computing  and  modifying  a  triangular  LU  factorization  of  the 
basis  matrix  B.  A  sparse  LU  factorization  of  the  basis  matrix  is  computed 
using  the  "bump  and  spike"  algorithm  of  Hellerman  and  Rarick  [Ref.  9: 
pp.  67-76],  which  is  updated  in  a  stable  manner  by  the  method  of  Bartels 
and  Golub  [Ref.  10:  pp.  266-268]. 
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In  order  to  extend  the  simplex  method  for  (LP)  to  (NLP), 
superbasic  variables  are  defined  in  addition  to  the  usual  basic  (dependent) 
and  nonbasic  (independent)  variables.  Both  basic  and  superbasic  variables 
may  possess  non-extremal  values  (between  their  respective  bounds).   In 
the  reduced-gradient  technique  employed  [Ref.  11:  pp.  97-131],  at  a 
given  iteration  there  are  NS  (Number  of  Superbasic)  superbasic  variables. 
They  are  free  to  move  in  any  desirable  direction  which  improves  the  value 
of  the  objective  function,  while  the  basic  variables  are  adjusted  to 
maintain  feasibility  with  respect  to  linear  constraints.   If  no 
improvement  can  apparently  be  made  with  the  current  set  of  superbasics, 
one  (or  more)  of  the  nonbasic  variables  is  selected  to  become  superbasic. 
This  increases  NS  and  the  process  is  repeated.   If,  at  any  time,  a 
basic  or  superbasic  variable  reaches  one  of  its  bounds,  that  variable 
becomes  nonbasic  and  NS  is  reduced  by  1.  By  the  usual  "pricing"  of  the 
nonbasic  columns,  Lagrange  multipliers  for  the  current  active  constraints 
are  obtained,  which  then  indicate  which  nonbasic  variables  (if  any) 
should  be  released  from  their  bounds.   If  required,  they  are  moved  from 
the  nonbasic  to  the  superbasic  set,  rather  than  from  nonbasic  to  basic  as 
in  the  more  conventional  simplex  method. 

A  stable  implementation  of  a  quasi -Newton  method  is  used  to 
optimize  over  the  superbasic  variables.  This  method  uses  a  triangular 
matrix  of  dimension  NS  to  approximate  the  reduced  Hessian  (a  suitably 

transformed  sub-section  of  the  matrix  of  second  derivatives, 

2 
(3  f/3x-3x-)).   In  large  problems  (i.e.,  NS  grows  as  large  as  100 

or  200),  the  data  region  required  by  the  quasi -Newton  method  becomes 

excessive  and  MINOS  automatically  substitutes  the  Fletcher-Reeves 
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Conjugate  Gradient  Method  [Ref.  12:  pp.  149-154],  which  consumes  relatively 
less  memory.  The  rate  of  convergence  of  the  algorithm  drops  significantly, 
but  must  be  accepted  due  to  storage  limitations. 

The  linear  constraint  problem  may  be  re-formulated  as: 


where: 
.     x 

.     x. 


N 


minimize 
subject  to 
P: 

given 


f(xN}   "  sobj 


[AN  AL  b   I] 
vf(xN)   =  g(xN) 


=  0,  b  < 


XL 

p 

s  J 


<  b 


=  the  nonlinear  variables   (those  that   are  directly 

involved  in  the  function  f(xN)); 
=  the  linear  variables   (the  remaining  part  of  x); 
=  the  right-hand  side   (RHS)   variable,  which  has  upper 

and  lower  bounds  of  -1.0; 
=  the  slack  or  logical  variables  (one  for  each  row  of  A); 


s  .  .  =  CiX.  +  Cg  (linear  objective  function  value); 


=  [AN  A.  ]  linear  coefficients,  partitioned  as  xN  and  x.  ; 
=  right  hand  side  (RHS),  composition  or  r_   and  r,  where: 

)F       ; 

o  =  <    r     ; 

r  or  r  ;  otherwise; 


if  ANxN  +  ALxL  >  r 
if  ANXN  +  ALXL  <  ? 


m    =  the  number  of  rows  in  A; 
n     =  the  number  of  columns  in  A; 
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.  nn    =  the  number  of  nonlinear  variables,  the  number  of  terms 

in  V 

.  NS    =  the  number  of  superbasic  variables. 

At  any  particular  stage,  the  n  +  1  +  m  columns  of  [A  b  I]  are 
implicitly  ordered  as  shown  in  Figure  1.  The  nonlinear  variables  may  end 
up  anywhere  in  B,S,  or  N. 

b.  Nonlinear  Constraints 

When  a  problem  contains  nonlinear  constraints,  MINOS  does  not 
necessarily  satisfy  the  nonlinear  constraints  until  an  optimal  solution 
is  achieved.  Therefore,  the  nonlinear  constraint  functions  may  need  to  be 
defined  outside  their  ranges. 

The  problem  must  be  expressed  for  MINOS  in  the  following 
standard  form: 

minimize      f  (x)  +  c  x  +  d  y  (nonlinear  objective)        (1) 
subject  to  f(x)  +  A,y  =  b-^  (nonlinear  constraints)   (2) 

ApX  +  A3y  =  b2  (linear  constraints)      (3) 


b  < 


where: 


J_  b   (bounds  on  variables) 
f(x)  = 


(4) 


fX(x) 


fmKx) 

and  the  functions  f(x)  should  be  smooth  and  have  computable  gradients. 
The  components  of  x  are  the  nonlinear  variables  and  must  preceed  the 
linear  variables  y  in  the  problem.  The  constraints  (2)  are  the  nonlinear 
constraints  and  must  appear  in  the  problem  before  the  linear  constraints 
(3).  The  general  constraints  may  contain  any  type  of  inequality,  and 
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m 


NS 


n  +  1  -  NS 


[A  b  I]  P  =  | 


B 


N 


Basic    Super-basic   Nonbasic 
where:  P  is  a  column  permutation  transformation  of  P 

Fig.  1.  MINOS  Tableau  Arrangement 


ranges  may  be  defined  for  the  constraints.  Upper  and  lower  bounds  (4) 
may  (and  should!)  be  specified  for  all  variables. 

The  solution  process  [Ref.  13]  consists  of  a  sequence  of 
"major  iterations;"  at  the  beginning  of  each  the  nonlinear  constraints 
are  linearized  at  a  current  point  x.  ,  approximated  to  first-order  by: 

f(x,  xk)  =  f(xk)  +  J(xk)(x  -  xk)  , 


which  can  be  written  as: 

f  =  fk  +  Jk(x  -  xk)  . 

i.  U  -i 

Here,  J(x)  is  the  Jacobian  matrix  whose  ij   element  is  af  (x)/ax.. 
The  objective  function  is  also  modified,  producing  the  following 
subproblem: 
minimize 


(5) 


f°(x)  +  cTx  +  dTy  -  xj(f  -  f) 


+  (l/2)p(f  -  f)T(f  -  f)  (quadratic  objective  (6) 

function) 


subject  to   f  +  A,y  =  b,     (linearized  constraints)      (7) 

(linear  constraints)         (8) 

(9) 


ApX  +  A^y  =  b~ 


b  < 


<  b    (bounds  on  variables). 
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The  objective  function  (6)  is  called  an  augmented  Lagrangian.  The 
vector  x.  is  an  estimate  of  the  Lagrange  multipliers  for  the  nonlinear 
constraints,  and  the  term  involving  p  is  a  modified  quadratic  penalty 
function.   If  desired,  the  Lagrangian  term  of  the  modified  objective 
function  may  be  set  to  zero  by  the  user.  The  penalty  parameter  p  may 
also  be  controlled  by  the  user.  The  problem  (6)-(9)  is,  of  course, 
stated  in  precisely  the  form  required  by  the  linearly  constrained  MINOS 
algorithm. 

A  flowchart  of  the  MINOS  NLP  solution  process  is  given 
in  Figure  2. 

If  (x.  ,  x. )  are  the  final  solution  and  multiplier  estimates 
from  the  k   subproblem,  convergence  is  assumed  to  have  occurred 
if  the  following  conditions  are  true: 

.  x.  is  an  optimal  solution  to  the  subproblem; 

.  x.  satisfies  the  nonlinear  constraints  within  a  specified 

tolerance; 
.  x.  is  not  substantially  different  from  \.    ,; 
.  x.+i  is  an  optimal  solution  to  its  subproblem; 

a  basis  change  did  not  occur  during  solution  of  subproblem  k  +  1; 
.  the  reduced  gradient  did  not  increase  significantly  during 
solution  of  that  subproblem. 

The  salient  point  is  that  x.  is  checked  for  feasibility 
and  then  the  final  point  x.  +  ,  is  checked  for  optimality.  Since 
normally  very  few  basis  changes  occur  on  the  final  subproblem  (ideally  none), 
the  solutions  will  be  virtually  identical  and  the  tests  for  feasibility 
and  optimality  will  have  been  applied  to  essentially  the  same  point. 
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Fig.  2.  MINOS  Iteration  Flowchart 
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2.  Code  Structure 

MINOS  is  intended  for  use  primarily  as  a  "stand-alone"  system 
which  solves  a  problem  or  a  sequence  of  problems  and  then  terminates; 
however,  MINOS  can  also  be  called  as  a  subprogram.  Figure  3  shows  the 
subroutine  structure  of  MINOS. 

a.  Stand-Alone  Structure 

Some  problem  description  and  code  tuning  are  provided  to 
MINOS  by  means  of  a  SPECS  file  which  contains  a  list  of  keywords  and 
values  to  define  run-time  parameters.  Most  of  the  data  for  a  problem 
is  provided  by  means  of  the  standard  MPS  Format  file.   If  the  problem 
contains  a  nonlinear  objective  function,  it  is  specified  (as  can  be  its 
gradient)  by  means  of  a  FORTRAN  subroutine  called  CALCFG  provided  by  the 
user.   If  the  problem  contains  nonlinear  constraints,  they  are  provided 
by  the  user  (along  with  their  gradients)  in  the  form  of  a  FORTRAN  sub- 
routine called  CALCON.  The  input  data  is  processed  in  the  following 
order: 

.  the  SPECS  file; 
.  the  MPS  file; 

.  a  basis  file  (allows  an  initial  basis  crash); 
data  read  by  CALCON  on  its  first  entry; 
data  read  by  CALCFG  on  its  first  entry; 

data  read  by  CALCFG  on  its  last  entry; 
data  read  by  CALCON  on  its  last  entry. 

The  MAIN  program  provides  a  single  array  of  storage  for  the 
code.  The  GO  subroutine  is  a  control  routine  which  calls  subroutine 
MINOS  for  each  problem  to  be  solved  in  the  input  stream.  This  routine 
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Fig.  3.  MINOS  Subroutine  Structure 
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can  be  modified  for  individual  applications.  Subroutine  MINOS  and  its 
sub-programs  communicate  with  the  input  files  supplied  to  the  code  by  the 
user  (MPS  and  SPECS)  and  generate  BASIS  and  SOLUTION  files.  MINOS  also 
contains  output  parameters  which  define  the  stopping  condition  for  the 
problem,  the  dimensions  of  the  problem,  and  the  positions  in  the  workspace 
array  where  various  subarrays  are  located. 

b.  Use  as  a  Subroutine 

Because  of  the  modular  nature  of  the  code,  it  is  relatively 
easy  to  modify  the  system  for  individual  applications.  Most  modifications 
to  tailor  the  system  would  be  to  the  control  routine  GO;  run-time 
parameters  and  constraint  information  would  still  be  input  via  the  MPS 
and  SPECS  files.  The  dotted  lines  in  Figure  3  show  how  a  Matrix  Generator 
and  Report  Writer  could  be  incorporated  into  the  system  to  be  used  in  any 
manner  convenient  to  the  user. 
3.  Documentation 

MINOS  documentation  consists  of  three  publications  [Ref.  4,  5,  6]. 
They  provide  such  complete  coverage  of  the  system  that  only  rarely  has 
access  to  the  MINOS  FORTRAN  source  code  been  required. 

a.  MINOS  User's  Guide 

This  document  is  for  the  original  MINOS  system,  which  did 
not  accommodate  nonlinear  constraints.  It  is  quite  thorough  and  provides 
some  of  the  mathematical  foundations  upon  which  the  code  is  built.  It 
has  one  section  devoted  solely  to  problems  with  nonlinear  objective 
functions.  The  input  data  required  by  the  user  is  also  thoroughly  docu- 
mented with  detailed  coverage  of  the  options  available  in  the  SPECS  file 
as  well  as  some  description  of  the  "MPS  Format"  used  by  the  MPS  file.  The 
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various  input/output  options  involving  the  basis  and  solution  files  are 
addressed  in  great  detail.  A  complete  breakdown  of  the  iteration  log 
output  is  given  to  allow  the  user  to  trace  solution  trajectory,  if 
required.  There  is  a  good  deal  of  coverage  of  hardware-dependent  matters, 
since  there  are  some  subroutines  that  are  computer  manufacturer  or 
machine  dependent.  Example  problems  are  also  given  with  complete  imple- 
mentation advice. 

b.  MINOS/AUGMENTED  User's  Manual 

This  manual  serves  as  an  adjunct  to  (not  a  replacement 
for)  the  User's  Guide.   It  is  intended  primarily  to  update  the  User's 
Guide  for  solving  problems  with  nonlinear  constraints,  but  contains 
additional  information  as  well.  Some  mathematical  overview  is  given  for 
the  nonlinear  constraint  case,  as  well  as  the  necessary  additional 
commands  in  the  SPECS  file  and  the  additional  subroutine  (CALCON)  required 
There  is  some  alteration  of  the  syntax  and  keyword  meanings  and  options 
from  the  (previously  described)  MINOS  User's  Guide,  and  the  coverage  is 
not  complete;  therefore,  both  documents  must  be  used  for  nonlinearly 
constrained  problems.  MINOS  can  most  readily  be  applied  by  synthesizing 
both  documents  to  create  a  "driver  template"  (such  as  those  used  by  the 
X  System)  for  input  files  and  function  subroutines;  this  shows  the  user 
most  available  options,  and  references  those  that  cannot  readily  fit 
in  the  "template"  format.  The  MINOS/AUGMENTED  Manual  also  contains 
sample  problems  with  complete  input  and  output  listings  which  provide  a 
useful  guide  to  the  new  user. 
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c.  MINOS  Distribution  Documentation 

This  document  serves  as  a  cover  letter  for  the  distribution 
of  the  code  via  magnetic  tape.  It  discusses  in  detail  the  machine- 
specific  requirements  for  successful  installation  of  MINOS  on  the  user's 
computer  system.  MINOS  is  currently  available  with  machine-dependent 
routines  tailored  for  Burroughs,  CDC,  DEC,  Honeywell,  IBM,  and  Univac 
computers.  The  documentation  also  describes  a  procedure  to  run  test 
problems  on  the  distributed  tape  which  allow  a  complete  check  of  the 
code.  Problems  6  and  11  of  this  study  are  included  as  test  problems  on 
the  MINOS  tape.  Some  minor  changes  that  have  been  made  to  the  system 
since  the  publication  of  the  two  previous  manuals  are  also  presented. 
4.  Implementation 

The  first  step  in  problem  solution  is  one  of  proper  problem 
formulation.  The  need  for  understanding  the  problem,  determining 
proper  scaling,  and  establishing  sensible  ranges  and  bounds  cannot  be 
overemphasized  and  will  be  discussed  more  fully  in  Section  III.   Ignoring 
these  steps  in  haste  will  invariably  haunt  the  analyst  in  the  long 
run. 

a.  CALCFG 

Subroutine  CALCFG  provides  MINOS  with  explicit  calculations 
of  the  nonlinear  objective  function  and  its  gradient.  There  is  an 
indicator  variable  in  the  calling  arguments  that  indicates  the  first, 
last,  or  some  intermediate  call  to  the  subroutine,  providing  it  an 
opportunity  to  make  intiial  or  final  calculations  if  required. 

MINOS  allows  substitution  of  numerical  difference  approx- 
imation for  the  analytical  gradient  of  the  objective  function. 
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b.  CALCON 

Subroutine  CALCON  serves  the  same  purpose  for  the  nonlinear 
constraints  as  does  CALCFG  for  the  objective  function.  However,  the 
option  for  differencing  of  the  constraints  in  lieu  of  analytic  gradients 
is  not  presently  available.  The  gradient  functions  must  be  explicitly 
provided. 

c.  SPECS  File 

The  SPECS  file  consists  of  80-column,  card-image  records 
that  provide  MINOS  with  the  parameters  of  each  problem  and  allow  the  user 
to  set  virtually  every  control  parameter  within  the  code.  Each  of  the 
card  records  contains  (in  free  format)  a  sequence  of  items  that  includes  a 
first  "keyword,"  an  optional  second  "keyword,"  and  a  number  representing 
the  value  of  the  parameter  identified  by  the  keywords. 

d.  MPS  File 

Constraint  matrix  data  and  labels  for  constraints  and  vari- 
ables are  provided  to  MINOS  via  an  MPS  file,  which  (as  the  name 
implies)  uses  standard  "MPS  Format"  [Ref.  3].   In  nonlinear  problems,  the 
only  restrictions  are  that  the  nonlinear  variables  must  occur  in  the 
left-most  columns  of  the  constraint  matrix,  and  the  non-linear  constraints 
must  be  the  top-most  rows  of  the  matrix.  Additionally,  an  optional 
INITIAL  bounds  set  may  be  specified  which  assigns  initial  values  (if 
available)  to  the  nonlinear  variables  in  the  problem. 

The  need  to  state  a  problem  formulation  with  proper  scale  for 
both  functions  and  variables  cannot  be  overemphasized. 
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e.  Basis  Files 

MINOS  provides  four  different  methods  for  loading  or  saving 
basis  representations.  These  options  are  invoked  by  use  of  records  in 
the  SPECS  file  and  allow  the  user  to  save  current  values/solutions  and 
reload  them  at  a  later  time  using  a  number  of  alternatives.  Because  of 
the  relatively  small  size  of  the  problems  in  this  study,  these  options 
were  not  extensively  used,  but  appear  to  be  well  thought  out  and  of  great 
potential  value  in  working  with  large  real-life  problems. 

f.  Machine  Dependent  Routines 

MINOS  was  developed  on  IBM  computer  systems,  and  although 
every  attempt  was  made  to  make  the  code  as  portable  as  possible  using 
FORTRAN,  some  machine  dependent  details  must  be  changed  in  order  to 
install  the  code  on  other  than  an  IBM  machine. 
5.  Output 

MINOS  provides  a  great  deal  of  flexibility  in  the  detail  of  its 
output  through  several  SPECS  file  options  as  well  as  the  opportunity  for 
using  a  tailored  report  writer, 
a.  Print  Level 

The  most  direct  control  of  output  is  via  the  PRINT  LEVEL 
option  of  the  SPECS  file.  This  command,  which  can  be  thought  of  as  a 
five-digit  binary  variable,  allows  the  user  to  individually  select 
any  one  or  all  of  the  following: 
.  invert  statistics; 
.  the  value  of  nonlinear  variables; 
.  the  Lagrange-multiplier  estimates  for  the  nonlinear  constraints; 
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.  the  values  of  the  nonlinear  constraint  functions; 
.  the  Jacobian  matrix. 

In  addition,  the  LOG  FREQUENCY  command  controls  how  often 
information  is  printed  from  the  iteration  log.  This  informs  the  user  at 
specified  intervals  of  the  status  of  the  optimization  effort, 
b.  Solution  File 

The  solution  file  can  be  printed  on  any  device  (determined  by 
a  SPECS  file  entry)  and  is  a  static  preformatted  report.   It  is  also 
designed  to  be  subsequently  read  from  disk  by  a  self-contained  program 
which  extracts  and  saves  the  values  required  by  the  user.  This  allows 
the  user  to  make  general  use  of  the  solution  without  first  tailoring  the 
entire  output  file  or  modifying  the  basic  code  with  a  tailored  report 
writer. 

6.  Debugging 

In  general,  MINOS  is  an  easy  code  to  work  with  as  long  as  the 
user  can  interpret  the  iteration  log  and  ancillary  printouts  in  the  more 
complex  cases. 

a.  Gradient  Checks 

The  requirement  for  explicitly  calculating  gradients  for 
the  constraints,  and  preferably  for  the  objective  function  as  well,  is  a 
nuisance  at  best;  but  it  can  prove  to  be  helpful  in  the  early  debugging 
phase  of  a  complex  problem.  MINOS  checks  the  analytic  gradient,  comparing 
the  function  and  gradient  information  provided  by  the  user  for  consistency, 
This  check  normally  uncovers  errors  of  coding  and/or  calculus,  or  gives 
the  user  at  least  some  hope  that  the  provided  functions  and  gradients  are 
correct. 
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b.  Error  Statements 

Although  the  diagnostic  statements  provided  by  MINOS  are 
explicit  and  normally  lead  directly  to  an  error,  a  few  are  not  documented 
in  the  support  literature  at  this  time  and  can  lead  to  some  confusion. 
Consulting  the  source  code  is  of  some  help,  since  it  is  very  well  docu- 
mented for  the  technically  qualified  reader.  However,  not  even  the 
source  code  review  has  been  able  to  pinpoint  the  cause  of  all  errors  and 
has  forced  the  user  to  occasionally  resort  to  more  traditional  FORTRAN 
error-checking  techniques. 

B.  X  SYSTEM 

XS  has  been  developed  since  1974  [Ref.  14]  as  a  general -purpose 
optimization  system  of  advanced  design  which  serves  both  as  a  prototype 
testbed  for  research  and  as  the  fundamental  computational  foundation 
of  many  application  packages  utilizing  optimization. 

XS  is  designed  to  solve  large-scale  optimization  problems,  with 
special  emphasis  on  mixed  integer  models.  Decomposition  features  are  of 
premier  importance  to  XS  at  an  aggregate  level  of  detail.  However,  the 
embedded  linear  programming  module  has  received  the  most  design  effort-- 
it  is  the  heart  of  XS  and  exhibits  many  singular  features  including: 

.  hyper-sparse  data  representation  [e.g.,  Ref.  15]; 

.  complete,  constructive  degeneracy  resolution  [e.g.,  Ref.  16: 
PP.  1-34]; 

.  basis  factorization  [e.g.,  Ref.  17:  pp.  91-110]; 

.  elastic  range  constraints  [Ref.  14]. 

The  nonlinear  feature  of  the  X  System  is  designed  for  use  with 
large-scale  models  exhibiting  some  functional  nonlinearity  (non-separable, 
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nonlinear  objective  and  constraints  are  admitted).  However,  XS  is 
designed  to  efficiently  solve  models  for  which  nonlinearity  is  relatively 
mild,  hopefully  involving  a  subset  of  model  variables  and  constraints. 
Large  refining  models  provide  an  ideal  nonlinear  paradigm  in  this  context. 

Design  criteria  for  XS  require  that  all  other  features  be  supported 
simultaneously  with  the  nonlinear  feature  (e.g.,  see  Problem  12  with 
generalized  upper  bound  (SUB)  factorization  and  mixed  integer  (MIP) 
variables  in  addition  to  nonlinear  features).  This  inflexible  rule 
derives  both  from  the  wide  variety  of  production  applications  currently 
using  XS,  as  well  as  from  the  research  philosophy  of  the  developers. 

The  basic  nonlinear  feature  of  XS  is  designed  to  encourage  reliable, 
error-free  problem  input  and  solution,  minimizing  the  total  response  time 
including  problem  preparation,  debug,  tuning,  and  interpretation  of 
output.  Accordingly,  gradient  functions  are  not  required,  nor  are  they 
even  suggested  as  an  attractive  option  to  any  but  the  most  skilled  user. 
Long  experience  has  shown  that  coordination  of  gradient  functions  with 
problem  functions  reliably  inflicts  more  frustration  than  all  other  steps 
in  solving  nonlinear  models!  The  X  System  is  designed  for  robust  perfor- 
mance with  automatic  numerical  difference  approximation  for  all  functions. 

Another  notable  nonlinear  feature  of  XS  is  the  pervasive  use  of 
robust  problem-independent  algorithm  controls,  such  as  a  dynamic  "trust 
region"  for  approximation  of  the  problem  functions,  a  conservative  ray 
search,  and  heuristic  internal  tolerances  to  stabilize  performance  in  the 
presence  of  "irregularities"  such  as  function  discontinuities  (common  to 
many  models,  especially  those  that  are  new  and  those  that  employ  spline 
approximations  [see  Section  III  .B]  from  imprecise  data  observations). 
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Also,  use  of  the  elastic  ranges  invites  elegant  formal  incorporation  of 
internal  estimates  for  approximate  data,  prioritizing  of  constraints,  and 
various  relaxation  methods. 

XS  completely  consists  of  open  FORTRAN  subroutines.  FORTRAN  IV 
(H  Extended)  [Ref.  2]  is  the  implementation  dialect  and  an  IBM  3033  the 
host  computer.  XS  is  installed  and  operating  on  other  systems  with 
compatible  arithmetic  architecture,  and  the  FORTRAN  subset  actually  used 
is  widely  accepted,  even  by  FORTRAN  77  [Ref.  18].   In-core,  out-of-core 
procedures,  if  invoked,  require  the  most  installation-specific  preparation 

Problem  input  may  include  any  combination  of  explicit  function 
definitions,  sparse  row  and/or  sparse  column  generators,  and  MPS  Format. 
No  tuning  parameters,  gradients  or  other  problem-specific  data  are 
required  other  than  some  adequate  (or  default)  estimate  of  sheer  problem 
dimensions  (rows  and  columns).  An  example  of  the  minimum  information 
necessary  to  invoke  the  algorithm  for  a  sample  problem  (Problem  9)  is 
provided  in  Figure  4. 

Input  may  use  any  of  several  standard  FORTRAN  subroutine  "templates" 
which  require  only  rudimentary  programming  skill.  Alternate  interface 
procedures  may  exercise  XS  directly  for  extremely  challenging  problems. 
All  relevant  tuning  defaults  and  tolerances  are  available  for  user 
override  via  a  simple  menu  in  FORTRAN  subroutine  form. 

XS  employs  an  (NLP)  technique  which  generates  a  sequence  of  local 
linear  programs  (LP),  each  from  previous  solution  and  gradient  estimates. 
Each  local  LP  determines  a  search  direction  based  on  a  first-order  local 
approximation  of  the  problem  functions  and  on  the  dual  variable  bounds 
(elastic  range  penalties),  forming  a  piecewise  linear  Lagrangian  gradient 
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Formulation: 

maximize 

*1  *  y2 

subject  to 

2      2 

h      +  y2 

(30)2   (23)2 

yx,  y2  >-o 

=  1 


Implementation: 

M    =1  (number  of  constraints) 

N    =2  (number  of  variables) 

5(1)  =  Y(1)**2/900.0D0  +  Y(2)**2/529.0D0   (constraint) 

GO  =  Y(l)  *  Y(2)  (objective  function) 

Fig.  4.  X  System  Problem  Generation  in  Explicit  Mode 

and  investigating  this  general  search  direction  with  a  local,  closed- 
interval  ray  search. 

XS  reliably  solves  NLP's  with  excellent  efficiency.  More  important, 
XS  can  be  tuned  for  particular  classes  of  problems  to  provide  unparalleled 
solution  speed.  However,  XS  employs  a  first-order  descent  method,  and 
one  would  expect  that  convergence  to  more  than  3-4  decimal  place  objective 
function  tolerance  will  occasionally  require  disproportionate  effort. 
The  designers  have  consciously  opted  for  fast,  easy,  absolutely  reliable 
convergence  to  3-place  precision,  a  tolerance  frequently  exceeding 
coefficient  data  resolution.  Although  much  better  solution  resolution  is 
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desirable,  and  routinely  achieved  for  amenable  problems,  it  is  not 
guaranteed  by  XS  without  some  experience  (tuning). 

For  those  cases  in  which  extreme  solution  tolerance  is  demanded, 
a  second-order  variation  is  available  which  augments  the  explicit  con- 
straints with  the  first-order  stationary  conditions.  This  method  has 
produced  excellent  results  [Ref.  19],  but  the  additional  problem  set-up 
time  is  seldom  economically  justified  in  lieu  of  simply  running  default 
first-order  XS  a  bit  longer.  (This  represents  a  classic  paradox  in 
evaluating  algorithms  and  their  actual  application.)  Given  the  additional 
set-up  effort,  robust  hybrid  first-  and  second-order  descent  algorithms 
are  easily  implemented  by  use  of  the  constraint  penalties  as  descent 
forcing  functions. 

1.  Algorithm 

The  X  System  requires  that   (NLP)  be  restated  elastically: 
(ENLP)       minimize  g  (y)  =  f(y)       +     z_a  +  zr 

subject  to  g(y)  +s+a     -  r  =  r 

ill y  £  6 

o  <_  s  <_  r  -  jr 
a,  r  _>  0 
or,  in  an  equivalent  Lagrangian  form: 

minimize     f(y)  +  z  (g(y)  -  r) 


subject  to   b  <   y  <:  b 


Zi  ;  if  g^y)  >  r  =  r. 


with        z-   =  I  -z_-    ;  if  g-{y)   <  r  =  r- 

0   ;  otherwise  . 
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We  see  immediately  that  z_,  z  are  interpreted  both  as  constraint  penalties 
and  as  linear  dual  bounds,  and  that  convexity  is  preserved  with  z_,  z  _>  0. 
Given  some  solution  for  (ENLP), 
b  <  y°  <  6  , 
a  local  elastic  linear  program  is  generated: 

(LELP)   minimize     vg  (y  ) Ay  +  z  {vg- (y  ) Ay  -  r) 
subject  to   b  <_  Ay  _<  6" 

•i     ;   if  vgi(y°)Ay  >  r  =  r.   -  gri (y°) 
with  z-   =  {-z-     ;   if  ?gri(y  )&y  <  r  =  r^   -  g^{y  ) 

0   ;  otherwise 
where  b,  6"  are  local  bounds  derived  from  b_,  6  and  enforced  as  a  local 
"trust  region"  over  which  the  functions  g   are  approximated  acceptably  by 
first-order  Taylor  series  and  z,  z  are  local  penalties  derived  from 
z_,  z  (and  optionally  other  artifacts  of  solution). 

The  complete  algorithm  performs  a  ray  search  on  the  piecewise 
linear  convex  (for  convex  functions  g)   descent  direction  Ay,  updates 
y  ,  and  generates  another  local  elastic  program  if  terminal  conditions 
are  not  satisfied  (see  Figure  5).   In  particular,  descent  is  assumed 
if: 

a).  The  latest  improvement  in  the  objective  function  is 
greater  than  a  (specified)  tolerance,  and 

b).  The  maximum  number  of  (specified)  iterations  has  not  been 
exceeded. 
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Initial  Solution 
b  <  y°  <  5 


> 


V 


General 
ENLP 


yes 


■> 


Solution 


V 


Linearized  LELP 
Find  Ay 


Ray  Search, 
Update 

0    0 
y  *  y  +  yAy 


Note:   Integer  detail  ommited  from  NLP  solution  process  for 
clarity. 


Fig.  5.  X  System  Iteration  Flowchart 
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Convergence  is  assumed  if  there  is  no  descent  for  a  (specified)  number 
of  iterations. 

At  each  iteration,  the  local  trust  region  is  determined  dynam- 
ically based  upon  (specified)  minimum  and  maximum  desired  step-length  (in 
y-units).   It  is  increased  in  size  if  descent  has  been  achieved,  decreased 
otherwise.   In  addition,  each  local  bound  is  modified  by  a  (specified) 
factor  if  oscillatory  behavior  is  observed. 
2.  Code  Structure 

The  X  System  is  primarily  designed  and  used  as  an  "open  sub- 
routine" system,  but  can  be  run  "stand-alone"  for  problems  of  specified 
structure.  Figure  6  shows  the  module  organization  of  XS. 

Default  tuning  parameters  are  defined  in  the  NUCLEUS.  For 
problems  of  given  structure,  a  "template"  is  prepared  consisting  of 
two  entry  points.  PROB  initializes  problem  generators  and  specifies  any 
non-default  initial  parameters;  FGE  provides  function  values  as  required 
by  the  problem  generators,  and  can  also  dynamically  change  any  parameter. 
Standard  problem  templates  include: 
.  Explicit  Mode  (FGE  provides  explicit  statements  of  the  problem 

functions  in  terms  of  y); 
.  Sparse  Mode  (FGE  provides  first-order  function  information  only 

once,  and  computes  nonlinear  terms  on  demand); 
.  MPS  Mode  (Sparse  Mode  with  MPS  input); 

.  Super-sparse  Mode  (PROB  and  FGE  communicate  directly  with  internal 
XS  data  structures) . 
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MAGEN, 
(etc.) 


PROB 
(Initialize  prob. 
generators,  e.g., 
MPS,  etc.) 

"Template" 

FGE 

(Function 
Generator) 

■> 


<r 


"> 


MAIN 
(NUCLEUS) 


±. 


PARM 
(Acquire 
Storage) 


\/ 


MP 
(Allocate 
Storage, 
Control ) 


V 


NL 

(Nonlinear 

Control ) 


±. 


GSM 
(Ray  Search, 

update  y  ) 


<—> 


GLOBAL 

(e.g., 

Decomposition) 


<r-> 


REPWRT 
(etc.) 


XS 
(Solve 
Local  MIP) 


Fig.  6.  X  System  Module  Structure 
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3.  Documentation 

User  documentation  for  XS  is  terse.  A  brief  summary  of  the 
system  and  examples  of  LP,  MIP,  and  NLP  formulations,  templates,  set-ups, 
and  solutions  are  given  [Ref.  20].  The  mathematical  foundations  are  set 
forth  in  [Ref.  21].  Finally,  the  designers  provide  an  internal  mainte- 
nance specification  and  logic  manual  with  full  current  implementation 
details  and  data  structures. 

Complete  documentation  for  XS  is  prepared  for  particular  produc- 
tion applications  [e.g.,  Ref.  22,  23].  Also,  user-friendly  interactive 
templates  permit  "classroom  use"  of  XS  in  real-time  with  very  little 
training.  However,  full  documentation  for  prototype  XS  is  not  likely  to 
appear  until  the  system  stops  improving. 

4.  Implementation 

Problem  preparation  for  XS  is  straight-forward.  First,  a  template 
should  be  selected  which  matches  the  requirements  of  the  problem  at 
hand. 

a.  Mode  Selection 

Explicit  Mode  --  Adequate  for  small  and  intermediate-sized 
problems,  very  easy  to  use,  but  a  bit  less  efficient  than  other,  more 
demanding  modes.  Efficiency  is  enhanced  if  nonlinear  constraints  and 
variables  are  identified  for  XS  in  a  contiguous  block. 

Sparse  Mode  --  Desirable  for  large  problems  and  relatively 
easy  to  use.  Efficiency  is  enhanced  if  non-null  coordinates  of  nonlinear 
constraints  and  variables  are  identified. 

MPS  Mode  --  Required  for  problems  from  outside  sources 
expressed  in  MPS  Format.  This  mode  also  provides  one  means  of  introducing 
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labels  for  functions  and  variables  using  proprietary  matrix  generators, 
identifying  non-null  coordinates  of  nonlinear  constraints  and  variables, 
and  providing  a  PENALTIES  section  in  an  easy  format.  However,  processing 
MPS  Format  files  is  relatively  expensive,  and  editing  requires  another 
file  with  revision  instructions. 

Super-sparse  Mode  --  Greatest  efficiency  and  risk.  Mandated 
only  by  huge  problems,  complex  generators,  or  other  uncommon  model 
curiosity. 

Once  the  proper  template  is  selected,  the  following  are 
assembled,  as  required. 

b.  PROB 

Subroutine  PROB  defines  initial  conditions  (connects  MPS 
files  as  necessary)  and  sets  up  problem  generators.  Default  tuning 
parameters  may  also  be  redefined. 

c.  FGE 

Subroutine  FGE  generates  function  values  for  a  given  solution, 
The  calling  discipline  provides  a  single  argument  which  invokes  complete 
function  evaluations,  only  nonlinear  function  evaluation,  only  non-null 
coordinate  function  evaluation,  or  only  non-null  nonlinear  function 
evaluation,  depending  upon  the  template  adopted.  Except  for  super-sparse 
mode,  the  solution,  base  (partially  evaluated)  function  values  and 
complete  function  values  are  all  available  in  precisely  the  organization 
(order)  specified  by  the  user.   In  super-sparse  mode,  basis  factorization 
and  permutation  vectors  may  be  exercised  directly  by  the  user,  as  well 
(not  suggested  for  the  timid!).  Tuning  parameters  may  be  redefined 
dynamically,  as  may  any  problem  element. 

45 


d.  MPS  File 

This  is  required  only  in  MPS  mode.  A  PENALTIES  section  may 
be  used,  as  may  an  associated  MPSEDIT  file.  Both  files  are  in  card  image 
form  with  80-character,  fixed-length  records. 

e.  SCRATCH  Files  (problem  independent  options) 

These  files  are  respectively  required  only  for  in-core, 
out-of-core  use,  for  online  storage  of  incumbent  solutions  in  non-convex 
and  decomposed  problems,  and  for  efficient  reinversion  of  the  basis. 
Unformatted  FORTRAN  Input/Output  is  used  with  the  attendant  file  defini- 
tion requirements. 

f.  CRASH  File  (problem  independent  options) 

This  is  required  for  basis  crash  options,  for  basis  save 
options,  and  for  efficient  reinversion.  Unformatted  FORTRAN  Input/ 
Output  is  used. 

g.  Machine  Dependent  Routines  (problem  independent  options) 

For  IBM  architecture,  XS  may  use  a  high-resolution  computation 
timer,  an  integer  overflow  arithmetic  hardware  interrupt  trap,  and  a 
dynamic  (GETMAIN)  storage  allocation.  These  routines  are  coded  in 
assembly  language. 
5.  Output 

Standard  output  includes  various  (specified)  levels  of  detail, 
ranging  from  absolutely  nothing  to  complete  logs  of  each  primitive 
operation  within  each  segment  of  each  minor  iteration.  Frequently  useful 
reports  include: 

Input  Summary  --  by  row  and  column  giving  labels  (if  any),  basis 
factorization  status,  selection  status,  non-null  element  counts,  maximum 
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magnitude  and  range  of  first-order  elements,  range  and  bound  status   and 
scale,   and  penalty  scale. 

Set-up  Summary  --  with  tuning  parameters,  dimension  keys,  host 
environment,  XS  vintage,  and  intended  problem  template. 

Major  Iteration  Log  --  with  complete  time  and  pivots,  solution 
value  (with  and  without  penalty  terms),  ray  search  resolution,  local 
gain,  trust  region  size,  and  descent  condition. 

Minor  Iteration  Log  --  (optional  frequency)  with  linear  information 
as  in  Major  Iteration  Log. 

Exception  Log  --  with  basis  crash  and  stability  recovery 
diagnostics. 

Solution  Summary  --  by  row  and  column  giving  labels  (if  any) 
and  complete  elastic  primal,  or  dual  solution  (or  edited  selected  subsets). 

Final  Summary  --  with  total  pivots  and  pivots  per  iteration, 
total  function  calls  and  calls  per  iteration  required  by  ray  search, 
integer,  and  differencing  operations,  respectively,  and  total  compute 
time. 

Solution  File  --  for  automated  real-time  solution  analysis 
[e.g.,  Ref.  24],  or  off-line  report  writing. 

In  addition,  a  report  writing  template  is  provided  for  use  in 
customizing  output  formats  and  report  content. 
6.  Debugging 

Nonlinear  models  fail  most  frequently  in  just  three  ways: 
programming  errors  in  the  problem  representation  fail  to  mate  it  compat- 
ibly with  calling  modules  and  data  structures,  or  model  errors  for  which 
a  correctly  coded  problem  representation  generates  an  incorrect  model,  or 
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solution  errors  yield  undesired  stationary  points  from  otherwise  correct 
models. 

Some  programming  errors  are  almost  completely  undetectable  by 
an  open  FORTRAN  subroutine  (e.g.,  subscript  range  errors,  argument 
list  omissions,  etc.).  Provision  of  standard  templates  is  the  best 
insurance  measure  for  these  perennial  annoyances.  Symptoms  of  such 
difficulties  are  usually  severe,  immediate,  and  fatal.  A  change  of  host 
language  would  ameliorate  this  situation,  but  benchmarks  of  FORTRAN 
competitors  have  not  yet  revealed  a  candidate  with  sufficient  execution 
efficiency  for  economic  use  with  XS. 

Since  XS  uses  no  analytic  gradients  (a  profound  advantage  in 
model  debugging),  one  need  usually  merely  verify  that  a  given  solution 
y  produces  function  values  correctly.  The  set-up  and  solution  sum- 
maries of  XS  usually  diagnose  further  trouble  very  quickly--the  elastic 
range  violations  reported  in  any  solution  focus  attention  on  likely 
candidate  functions  for  further  inspection. 

Of  course,  XS  can  minimize  an  objective  function  intended  for 
maximization,  and  can  produce  stationary  points  for  completely  incorrect 
formulations  and  implementations;  however,  these  are  intellectual  issues 
for  the  modeller  to  ponder  more  than  bugs  in  the  classic  sense.  No 
debugging  aid  can  guarantee  reliable  solution  of  a  model  which  is  not 
understood  by  the  user. 

Interpretation  of  terminal  solutions  sometimes  reveals  that 
a  local  stationary  point  has  been  selected.  XS  provides  an  alternate 
procedure  for  problems  suspected  of  such  behavior,  whereby  the  objective 
function  is  included  with  the  constraints,  and  its  penalties  used  to 
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enforce  successive  level  set  descent  at  each  major  iteration  (e.g., 
Problem  4  for  which  XS  has  found  three  stationary  points).  Non-convex 
problems  usually  also  require  that  the  ray  search  be  tuned  for  extra 
effort. 

Some  tuning  advice  is  always  a  valuable  part  of  solution  debugging 
For  variables,  XS  uses  a  zero  level  and  infinity  value  which  may  be 
altered.  The  trust  region  is  used  when  unruly  nonlinear  function  behavior 
(misleading  first-order  approximation)  is  predicted,  or  as  an  additional 
safeguard  which  costs  little.  A  distortion  factor  is  provided  by  which 
oscillatory  behavior  detected  in  successive  solutions  is  damped.  (This 
is  no  substitute  for  conjugate  directions  or  second-order  information, 
but  serves  well  in  practice  as  an  effective  heuristic). 

A  zero  tolerance  is  used  for  functions,  and  may  be  altered. 
XS  also  requires  an  objective  function  tolerance  for  nonlinear  convergence 
(and  for  integer  convergence),  as  well  as  penalties  for  violation  of  each 
constraint  range.  Thus,  the  modeller  must  express  a  priori  in  terms  of  a 
single  objective  function  the  consequences  of  violating  feasibility  and 
optimality. 

Default  values  for  all  tuning  parameters  are  necessarily  scale 
dependent  and  require  reasonably  well-scaled  models  for  reliable  perfor- 
mance.  In  this  sense,  the  X  system  is  rather  unforgiving  of  users  who  do 
not  express  their  models  completely  and  carefully. 
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III.   EXPERIMENTAL  METHOD  AND  RESULTS 

A.  METHOD 

In  preparation  for  this  study,  a  copy  of  the  documentation 
[Ref.  4,  5,  6]  and  code  for  MINOS  were  received  from  Saunders  (circa  May 
1980,  via  G.  Brown  from  a  Vienna  meeting  with  Saunders).  The  code  was 
received  on  magnetic  tape  which  contained  the  system  source  code  (both 
machine  dependent  and  portable),  test  problem  source  code,  assembler 
language  source  code,  object  module,  and  executable  load  module.  The 
machine  dependent  routines,  assembler  code,  and  the  object  modules  were 
all  compatible  for  the  IBM  system  on  which  this  test  was  run.  The 
OPTIMIZE  (3)  load  module  from  the  tape  was  not  used,  but  replaced  with 
one  compiled  at  the  Naval  Postgraduate  School  with  the  OPTIMIZE  (2) 
option  [Ref.  2].  (OPTIMIZE  (3)  is  not  available  at  the  Naval  Postgraduate 
School  at  this  writing.) 

The  X-system  was  already  in  place  on  the  Naval  Postgraduate  School 
computer  and  was  used  in  situ.   Its  working  "documentation"  [Ref.  20]  was 
also  available. 

Because  of  the  complete  MINOS  documentaion  and  the  availability  of 
several  test  problems  already  set  up  for  immediate  execution,  all  problems 
were  initially  prepared  for  MINOS  and  then  converted  to  XS  (except  as 
noted).  This  procedure  required  a  greater  initial  outlay  of  effort  (See 
Section  II. B)  because  of  MINOS'  requirement  for  gradient  information  not 
required  by  XS;  however,  the  consistency  checks  make  internally  by  MINOS 
of  the  function  and  gradient  routines  served  as  a  troubleshooting  tool  to 
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ensure  the  functions  had  been  correctly  coded.  As  a  result,  the  conver- 
sion of  each  problem  from  MINOS  to  XS  took  less  than  an  hour  in  most 
cases  using  the  computer  system  file  editor,  once  the  problems  were 
successfully  run  with  MINOS.  The  converse  was  not  true,  however,  as  the 
gradient  functions  required  considerable  time  and  effort  to  derive  and 
implement  for  MINOS. 

Both  systems  are  presently  being  developed,  supported,  and  used 
by  their  authors  on  IBM  3033' s.  Therefore,  neither  system  has  been  given 
an  advantage  by  the  computer  used  to  support  the  evaluation. 

B.   PROBLEM  FORMULATION  COMMENTS 

Although  "easy"  problems  can  be  solved  with  brute  force  by  either 
system,  large  or  tricky  problems  require  intelligent  problem  formulation 
for  either  system  to  be  successful.  Among  the  most  vital  guidelines  to 
consider  are  the  following. 

The  analyst  must  UNDERSTAND  THE  NONLINEAR  PROBLEM  in  order  to  success- 
fully solve  it  (or  in  order  to  assimilate  the  answer).  This  can  be  a 
major  hurdle  when  solving  someone  else's  nonlinear  problem  formulation  or 
test  problem  if  inadequate  documentation  exists  to  set  up  the  problem  in 
a  format  that  is  compatible  with  the  optimization  code  being  used. 
Although  true  for  both  optimization  systems,  in  the  case  of  XS  this  is 
absolutely  vital.  A  good  deal  of  the  power  of  XS  lies  in  the  use  of  its 
"elastic"  penalties  on  the  constraints.  If  the  problem  is  not  fully 
understood,  these  penalties  cannot  be  set  intelligently  and  at  best  the 
power  of  XS  is  diminished,  because  the  penalties  require  formal  transfor- 
mation by  the  modeller  of  infeasibi lities  to  units  of  the  objective 
function.  At  worst,  XS  may  not  be  able  to  solve  the  user-stated  problem 
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at  all  since  it  views  the  elastic  formulation  as  "the  problem."  This  can 
be  a  major  hurdle  when  solving  someone  else's  nonlinear  problem  formulation 
or  test  problem  if  inadequate  documentation  exists  to  set  up  the  problem 
in  a  format  that  is  compatible  with  the  optimization  code  being  used. 
There  is  a  tendency  in  many  problem  formulations  to  overlook  points 

within  the  allowable  variable  bounds  that  generate  singularities  in  the 

1  2 

problem  functions.  By  Murphy's  Law  and  O'Toole's  Corollary,  it  is  a 

virtual  certainty  that  any  code  will  attempt  to  evaluate  the  functions  at 

a  singularity  if  proper  precautions  are  not  taken  by  the  analyst. 

These  precautions  may  be  in  the  form  of  restricted  bounds  on  the  variables 

that  were  not  included  in  the  original  problem  specification  or  in  coding 

within  the  function  generator. 

For  example,  in  the  region  of  a  singularity  a  spline  function  may 
be  substituted  for  the  function  of  interest  at  some  point  such  that: 

.  Endpoints  are  satisfied  by  the  composite  function; 

.  The  composite  function  is  continuous; 

.  The  composite  function  has  a  continuous  derivative; 

.  The  composite  function  is  a  "reasonably"  close  approximation  of 
the  function  and  its  gradient  (e.g.,  a  "majorizing  function"). 

In  the  same  vein,  reasonable  bounds  for  the  variables  should  be 
specified  to  prevent  excessive  run  times  with  either  system.  Even  if  the 
problem  has  been  initially  specified  with  unbounded  variables,  the  intelli- 
gent analyst  can  restrict  the  problem  to  reasonable  bounds.  There  are 


If  anything  can  go  wrong,  it  will  ...  at  the  worst  possible 
moment  .  .  .  and  in  the  way  so  as  to  maximize  the  damage. 

2 
Murphy  was  an  optimist. 
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few  real-life  problems  for  which  an  infinite  value  for  a  variable 
makes  sense. 

Both  systems,  in  fact  any  useable  nonlinear  optimization  code,  are 
sensitive  to  scale  and  require  that  the  analyst  formulate  the  problem  in 
a  reasonable  manner.   In  general,  the  data,  variables,  and  functional 
values  should  be  as  close  to  1.0  as  possible.  Due  to  efficient 
specializations  of  both  algorithms  for  linear  (or  nearly  linear)  con- 
straints, it  is  worth  a  good  deal  of  effort  by  the  analyst  to  minimize 
the  nonlinearities  in  the  problem  by  appropriate  transformations.  A 
nonlinear  objective  function  is  usually  preferable  to  nonlinear  constraints 
(if  a  choice  exists  via  problem  transformations). 

It  is  mandatory  with  XS  and  strongly  recommended  with  MINOS  that  any 
initially  specified  solution  lie  within  the  bounds  of  the  variables. 

C.   PROBLEM  DESCRIPTIONS 

The  first  eight  of  the  test  problems  had  been  used  for  a  previous 
study  [Ref.  25]  and  were  collected  from  a  variety  of  sources.  The  first 
six  problems  are  listed  in  Himmelblau  [Ref.  26:  pp.  395-425]  and  the 
original  source  author/developer  for  each  is  listed.  The  next  two  are 
adaptations  of  an  inventory  model  [Ref.  27]  and  an  entropy  model  [Ref. 
28]  which  were  specifically  designed  to  illustrate  real-world  problems 
with  few  constraints  but  many  independent  variables,  albeit  with  a 
relatively  small  number  of  variables  compared  with  the  capabilities  of 
the  two  codes  under  test.  The  ninth  problem  was  developed  by  the  author 
during  course  work  at  the  Naval  Postgraduate  School. 

Appendix  A  provides  the  mathematical  problem  statement,  problem 
data,  starting  points,  and  an  initial  and  final  solution  from  each  code 
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for  all  problems  except  Problem  11:  the  formulation  of  ETAMACRO  provided 
as  a  test  problem  for  MINOS  by  its  developers  did  not  exactly  match  the 
description  of  [Ref.  29].  Time  considerations  precluded  resolution  of 
the  discrepancies,  and  although  the  MINOS  test  problem  formulation  was 
used,  neither  formulation  is  presented  here. 

The  remaining  four  problems  were  not  solved  by  both  codes  either  due  to 
sheer  time  constraints,  or  in  the  case  of  Problems  12  and  13,  because  integer 
variables  cannot  be  accommodated  by  MINOS. 

The  alternate  starting  points  for  problems  1-8  were  taken  from  Waterman 
[Ref.  25:  pp.  56-91]. 

1.  Problem  1  (Himmelblau  6) 

3 
This  problem  [Ref.  30  ]  is  an  example  of  determining  the  chemical 

composition  of  a  complex  mixture  under  conditions  of  chemical  equilibrium. 

It  contains  45  independent  variables  and  16  linear  equality  constraints. 

MINOS  returned  a  solution  very  quickly,  although  the  solution  from  the 

second  alternate  starting  point  differed  from  the  first  two  starting 

points.  XS  took  a  bit  longer  to  reach  a  solution  from  all  three 

starting  points;  however,  the  objective  value  was  improved  over  the  MINOS 

solutions  and  the  three  solutions  agreed  with  only  minor  variations. 

2.  Problem  2  (Himmelblau  4A) 

This  is  also  a  chemical  equilibrium  problem  which  had  been 
redefined  in  the  Himmelblau  study  from  a  problem  originally  formulated 
and  solved  by  Dantzig,  Johnson,  and  White  [Ref.  31:  pp.  751-755]  and 
discussed  by  Bracken  and  McCormick  [Ref.  32:  pp.  46-49].   It  contains  10 
independent  variables  and  3  nonlinear  equality  constraints.  Both  codes 


3 
Reference  not  viewed  by  author. 
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obtained  essentially  identical  solutions  from  all  three  starting  points 
in  about  the  same  amount  of  time. 

3.  Problem  3  (Himmelblau  18) 

This  problem  was  formulated  by  the  Shell  Development  Company 
for  the  original  Colville  study  [Ref.  33:  p.  22]  and  consisted  of  15 
independent  variables  and  5  nonlinear  inequality  constraints.  MINOS 
returned  a  slightly  better  solution,  though  both  codes  returned  solutions 
from  all  three  starting  points  in  approximately  the  same  amount  of 
time. 

4.  Problem  4  (Himmelblau  16) 

This  problem  [Ref.  34  ]  maximizes  the  area  of  a  hexagon  in 
which  the  maximum  diameter  is  unity.  There  are  nine  independent  vari- 
ables, 13  nonlinear  inequality  constraints,  and  a  lower  bound  of  zero  on 
one  variable.  This  problem  contains  many  local  stationary  points  and  its 
solution  difficulty  is  not  reflected  in  the  problem  summary  in  Section  0 
of  this  chapter.  Both  codes  required  a  little  tuning  in  order  to  avoid 
local  minima.  To  verify  both  MINOS  and  XS  solutions,  an  XS  option  was 
invoked  which  uses  an  alternate  formulation  of  the  problem  using  the 
original  objective  function  as  a  constraint,  replacing  it  with  an  implicit 
objective  function.  (Recall  that  XS  implicitly  uses  penalties  in  its 
objective  function.)  This  constraint  value  then  provides  the  actual 
objective  function  value  in  the  final  solution.  Although  the  final 
results  achieved  for  XS  using  the  original  formulation  exceeded  the 
alternate  method  in  speed,  alternate  formulation  appears  to  be  very 


4 
Reference  not  viewed  by  author. 
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robust  and  seems  to  be  a  useful  tool  with  much  to  recommend  it  for 
XS  used  on  problems  containing  multiple  local  starting  points. 

5.  Problem  5  (Himmelblau  20) 

5   - 
This  problem  [Ref.  35  ]  contains  a  linear  objective  function,  24 

independent  variables,  12  nonlinear  equality,  two  linear  equality,  and 

six  nonlinear  equality  constraints.  The  independent  variables  also  are 

bounded  to  positive  values.  Both  codes  returned  identical  solutions  from 

all  three  starting  points,  however,  XS  required  more  time  to  reach  the 

solution. 

6.  Problem  6  (Himmelblau  23--continuous  relaxation) 

This  is  a  weapon  assignment  problem  formulated  by  My  lander  [Ref. 
36  ]  and  presented  by  Bracken  and  McCormick  [Ref.  32:  pp.  22-26] 
with  100  independent  variables,  a  nonlinear  objective  function,  12  linear 
constraints  and  zero  lower  bSunds  for  all  variables.  This  problem  was 
formulated  for  XS  using  the  GUB  option.  Both  codes  returned  approximately 
equal  objective  function  values  from  all  three  starting  points,  however, 
the  returned  variable  values  were  quite  diverse,  which  comfirmed  previously 
reported  results  [Ref.  25,  26].  XS  required  somewhat  longer  to  reach  the 
solution. 

Since  it  was  suspected  that  the  variability  in  reported  results 
might  be  due  to  the  integer  nature  of  the  variables  in  the  original 
formulation  which  had  been  approximated  with  continuous  variables  by  all 
previous  solution  attempts,  the  problem  was  reformulated  with  integer 
(binary)  variables  using  XS  and  solved  again  as  Problem  12  below. 


5 
Reference  not  viewed  by  author 
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7.  Problem  7  (Waterman  7) 

This  problem  was  adapted  by  Waterman  [Ref.  25]  from  an  inventory 
model  created  by  Choe  and  Schrady  [Ref.  27:  pp.  451-463].  The  first  50 
variables  represent  the  reorder  quantity  for  50  inventory  items  and  the 
next  50  variables  represent  the  reorder  points  for  the  same  50  items. 
The  problem  contains  one  linear  and  one  nonlinear  inequality  constraint 
and  50  lower  bounds  on  the  variables.  This  was  the  most  difficult 
problem  solved  by  both  codes.  MINOS  returned  approximately  the  same 
solutions  from  all  three  starting  points,  however  much  more  time  was 
required  from  the  alternate  starting  points.  MINOS  could  not  achieve  a 
solution  from  the  second  starting  point  with  the  conjugate  gradient 
option,  but  achieved  much  quicker  solutions  from  the  remaining  two 
points.  XS  did  not  return  as  good  a  solution  from  any  of  the  starting 
points  and  also  required  more  time  to  solve  the  problem. 

8.  Problem  8  (Waterman  8) 

This  problem  was  adapted  from  an  entropy  model  proposed  by 
Scott  [Ref.  28:  pp.  204-211].  The  nodes  depict  46  population  centers 
connected  by  a  transportation  network,  represented  by  the  connecting 
arcs.  Using  a  congestion  cost  function,  the  model  yields  an  equilibrium 
solution  that  identifies  node  populations  as  entropic  functions  of  the 
total  cost  of  transportation  to  a  central  place  of  work.  The  problem  has 
46  independent  variables,  one  nonlinear  inequality  and  one  linear  equality 
constraint,  and  46  lower  bounds  on  the  variables.  MINOS  returned  the 
same  solutions  from  all  three  starting  points.  Solution  times  for  both 
systems  changed  significantly  with  starting  points. 
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9.  Problem  9  (Dean  9--Game) 

This  is  an  example  of  a  small  problem  developed  from  a  classroom 
exercise  to  serve  as  an  example  of  how  quickly  problems  of  a  very  basic 
nature  can  be  solved  using  the  codes  as  time-saving  tools.  The  problem 
has  one  nonlinear  equality  constraint  and  two  independent  variables. 
Both  codes  found  the  solution  from  all  three  starting  points  in  comparable 
amounts  of  time.  It  took  less  than  an  hour  (clock  time)  from  start  of 
initial  problem  formulation  until  solutions  were  available  from  both 
codes. 

10.  Problem  10  (Dean— Sortie) 

This  problem  is  from  current  work  being  done  by  the  U.S.  Air 
Force  based  on  a  model  done  by  Clasen,  Graves,  and  Lu  [Ref .  37].   It 
contains  81  linear  constraints  and  793  independent  variables.  Because  of 
time  constraints  and  an  unsuitable  data  format  for  direct  input  to  MINOS, 
the  problem  was  only  solved  using  XS.  The  GUB  feature  of  XS  was  also 
used  on  this  problem.  The  results  are  given  to  provide  a  measure  of 
comparison  of  this  relatively  large  problem  with  the  other  smaller  test 
problems.  Also,  the  run  time  for  this  problem  on  the  host  computer 
using  the  Air  Force's  present,  but  dated,  solution  algorithm  is  321 
seconds. 

11.  Problem  11  (Dean  — ETAMACRO) 

This  is  Manne's  energy  problem  ETAMACRO  [Ref.  29:  pp.  1-45]  and 
is  included  on  the  MINOS  distribution  tape  as  a  test  problem.   It  contains 
401  linear  constraints  and  688  independent  variables.  Because  the  model 
was  formulated  in  close  conjunction  with  the  developers  of  MINOS,  the 
formulation  is  very  compatible  with  that  code's  structure.  However,  it 
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was  not  suitably  formulated  to  exploit  the  elastic  features  of  XS  and 
although  the  problem  was  solved  in  its  linear  form  by  XS,  time  constraints 
precluded  a  complete  reformulation  which  would  have  been  necessary  in 
order  to  solve  the  problem  efficiently  on  XS.   It  is  presented  here  because 
of  its  relatively  large  size  in  comparison  to  the  other  test  problems,  in 
order  to  give  some  indication  of  MINOS'  performance  on  large  problems. 

12.  Problem  12  (Himmelblau  23--Integer  Nonlinear) 

This  is  a  reformulation  of  Problem  6  with  integer  (binary) 
variables  instead  of  the  continuous  approximations  used  in  Problem  6. 
XS's  GUB  option  was  also  exercised  on  this  problem.  Because  of  the 
integer  nature  of  the  variables,  the  problem  was  only  run  on  XS.  this 
problem  has  never  before  been  solved  as  a  nonlinear  integer  model. 

13.  Problem  13  (Dean--Integer  Nonlinear) 

This  problem  [Ref.  38:  pp.  519-536]  was  submitted  by  Schmit 
to  Graves  and  is  an  example  of  a  current  engineering  design  problem  that 
is  highly  nonlinear  and  contains  integer  variables.  It  has  48  nonlinear 
inequality  and  three  linear  inequality  constraints  and  12  independent 
variables,  of  which  8  are  integer  (binary).  Because  of  the  integer 
variables,  the  problem  was  solved  only  with  XS,  but  is  listed  as  an 
example  of  a  nonlinear  integer  problem  solution  by  XS. 

D.   PROBLEM  SUMMARY 

This  problem  summary  lists  results  for  MINOS  using  the  default 
(on  small  problems)  quasi-Newton  method  as  well  as  the  conjugate  gradient 
option  normally  used  only  for  large  problems.  This  is  done  to  present 
a  complete  comparison  of  the  two  codes  using  their  respective  large-scale 
algorithms.  XS  does  not  provide  an  automatic  second-order  descent  method 
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option  (although  [Ref.  19]  presents  the  groundwork  for  such  an  option 
without  the  "elastic"  framework)  since  it  would  be  of  little  use  with  the 
large  problems  for  which  XS  is  designed. 
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CPU  time 

2.67 

3.21 

3.22 

XS  (compact  template) 

Obj.  Fn. 

-1734.5 

-1735.0 

-1734.6 

Linearizations 

38 

31 

33 

Pivots 

2424 

2194 

2390 

Fn.  Evals. 

4068 

3333 

3542 

CPU  time 

3.85 

3.22 

3.51 

Problem  7  (Waterman  7) 

MINOS 

(quasi-Newton) 

Obj.  Fn. 

80.744 

80.727 

80.727 

Linearizations 

11 

33 

21 

Pivots 

373 

1227 

784 

Fn.  Evals. 

2087 

6927 

4231 

CPU  time 

28.03 

70.55 

54.67 

(Conj.  Grad.) 

Obj.  Fn. 

80.744 

80.727 

Linearizations 

19 

No 

15 

Pivots 

662 

Solution 

507 

Fn.  Evals. 

4657 

4041 

CPU  time 

14.64 

12.53 

XS  (compact  template) 

Obj.  Fn. 

85.231 

80.726 

90.604 

Linearizations 

54 

144 

117 

Pivots 

4489 

14088 

9207 

Fn.  Evals. 

5915 

15283 

12798 

CPU  time 

26.31 

67.68 

57.32 
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Primary       Alternate      Alternate 
Solution       Solution       Solution 


Problem  8  (Waterman  8) 


MINOS 

quasi -Newton) 

Obj.  Fn. 
Linearizations 
Pivots 
Fn.  Evals. 
CPU  time 

-3 

.4685 

6 

148 

718 

4.09 

-3.4685 

9 

231 

1163 

6.41 

-3.4685 

8 

146 

715 

4.06 

(Conj.  Grad.) 
Obj.  Fn. 
Linearizations 
Pivots 
Fn.  Evals. 
CPU  time 

-3 

.4685 

6 

163 

991 

2.79 

-3.4685 

8 

240 

1417 

4.04 

-3.4685 

5 

142 

884 

2.48 

XS  (compact  tempi  at 

e) 

Obj.  Fn. 
Linearizations 
Pivots 
Fn.  Evals. 
CPU  time 

-3 

.4519 

13 

787 

1225 

3.03 

-3.4689 

40 

1842 

3871 

8.02 

-3.4643 

12 

676 

1174 

2.57 

Problem  9  (Dean 

9- 

Game) 

MINOS 

(quasi -Newton) 

Obj.  Fn.  345.00  345.00  345.00 

Linearizations  6  5  5 

Pivots  12  8  6 

Fn.  Evals.  52  39  37 

CPU  time  0.05  0.03  0.03 

(Conj.  Grad.) 

Obj.   Fn.  345.00  345.00  345.00 

Linearizations  6  5  5 

Pivots  12  8  6 

Fn.   Evals.  52  39  37 

CPU  time  0.04  0.03  0.02 

XS  (compact  template) 

Obj.   Fn.  344.94  342.59  345.01 

Linearizations  33  3  10 

Pivots  63  3  14 

Fn.   Evals.  293  44  87 

CPU  time  0.15  0.01  0.05 
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Primary 
Solution 


Problem  10  (Dean  10— Sortie) 


XS  (super-sparse) 

Obj.  Fn. 
Linearizations 
Pivots 
Fn.  Evals. 
CPU  time 

2.0087D+6 

3 

977 

2393 

8.60 

Problem  11  (Dean 

11  — ETAMACRO) 

MINOS 

(quasi-Newton) 

Obj.  Fn. 

Linearizations 

Pivots 

Fn.  Evals. 

CPU  time 

1337.7 

1 

1331 

1926 

41.36 

(Conj.  Grad.) 
Obj.  Fn. 
Linearizations 
Pivots 
Fn.  Evals. 
CPU  time 

1337.7 

1 

32640 

68868 

1028.05 

Problem  12  (Himme 

ilblau 

23— Integ 

XS  (compact  template) 

Obj.  Fn. 
Linearizations 
Integer  Linear. 
Pivots 
Fn.  Evals. 
CPU  time 

-1734.2 

30 

3 

285 

211 

4.41 

Problem  13  (Dean 

13— 

Integer  No 

XS  (compact  template) 

Obj.  Fn. 
Linearizations 
Integer  Linear. 
Pivots 
Fn.  Evals. 
CPU  time 

0.12000 

6 

3 

63 

145 

0.55 
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IV.   CONCLUSIONS 

A.  ALGORITHM  CAPABILITIES 

1.  Type  of  Problems 

MINOS  is  capable  of  reliably  solving  both  the  general  (LP)  and 
(NLP)  problems  with  any  combination  of  linear  and  nonlinear  constraints, 
but  cannot  accommodate  integer  variables. 

XS  has  the  same  capabilities  as  MINOS  with  the  addition  of  the 
use  of  integer  variables.  XS  can  also  employ  decomposition  and  basis 
factorization. 

2.  Growth  Possibilities 

There  is  no  inherent  maximum  problem  size  for  either  MINOS 
or  XS.  For  sheer  capacity,  they  are  both  limited  by  the  computer  memory 
requirements  for  their  working  arrays.  However,  as  noted  in  Section 
II. A. 1  for  MINOS,  in  large  problems  (when  the  number  of  superbasic 
variables  exceeds  100  or  200)  the  shift  to  the  conjugate  gradient  algo- 
rithm, which  consumes  less  memory,  results  in  a  significant  decrease  in 
the  theoretical  rate  of  convegence  of  the  algorithm  [Ref.  5:  p.  10]. 

B.  CPU  TIME 

MINOS  was  a  bit  quicker  in  solving  many  of  the  problems  of  this 
study.  While  XS  yields  2-3  decimal  place  precision  in  the  objective 
function  faster  than  MINOS,  MINOS  usually  prevails  in  3-5  place  efficiency 
(even  when  using  first-order  conjugate  gradient  options). 
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C.  STORAGE  REQUIREMENTS 

For  linear  programs  containing  m  general  constraints,  roughly 
100(m) -bytes  of  memory  are  required  for  workspace  by  MINOS.   If  there  are 
many  nonlinear  variables,  additional  memory  may  be  required.  This 
workspace  size  may  be  adjusted  by  changing  the  size  of  one  array  in  the 
main  program  for  MINOS  or  by  a  non-FORTRAN  routine  that  allocates  storage 
at  run-time.  The  choice  of  method  is  machine-dependent  and  guidelines 
are  provided  to  the  user  in  the  documentation  [Ref.  4]. 

XS,  used  strictly  in-core,  requires  a  region  of  approximately 
56(MN)  +  8(NR)  +  200K-bytes  where  MN  is  the  total  rows  +  cols,  and  NR  is 
the  size  of  the  distinct  real  value  pool.  Storage  requirements  for 
nonlinear  problems  known  to  this  writer  are  not  a  significant  considera- 
tion for  XS,  or  for  MINOS. 

D.  NUMBER  OF  ITERATIONS 

Each  major  iteration  of  MINOS  creates  a  local  linearization  of 
the  nonlinear  program,  and  then  solves  it  after  addition  of  a  quadratic 
(augmented  Lagrangian)  objective  function.  XS  simply  solves  local 
linearizations  (with  augmentation  of  the  linear  penalty  function  and 
local  trust  region).  MINOS  usually  requires  less  of  its  iterations  than 
does  XS,  but  evidently  works  harder  on  each. 

Pivots  (minor  iterations)  for  MINOS  represent  classical  basis  exchanges 
(a  superbasic  variable  becomes  basic,  replacing  a  basic  variable  which 
becomes  nonbasic).  XS  pivots  are  counted  in  several  varieties,  including 
logical  cases  in  which  no  basis  change  or  objective  function  improvement 
takes  place  but  the  logical  composition  of  the  solution  changes.  XS 
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usually  requires  more  of  its  pivots  than  does  MINOS,  but  apparently 
executes  each  of  them  faster. 

Iteration  counts  do  not  appear  to  be  adeuate  general  measures  of 
efficiency  for  dissimilar  optimization  algorithms,  or  even  alternate 
implementations  of  the  same  algorithm. 

E.   NUMBER  OF  FUNCTION  EVALUATIONS 

For  the  objective  vunction  precision  specified  in  these  tests, 
MINOS  has  generally  required  less  function  (/gradient)  calls  than  XS. 
This  difference  can  be  crucial  for  problems  in  which  function  generation 
requires  the  vast  majority  of  computing  effort.  However,  a  paradox  is 
apparent  in  that  such  problems  often  have  no  closed-firm  derivatives: 
desirable  for  (quasi -Newton)  second-order  algorithms.   In  most  cases, 
XS  provides  2-3  decimal  place  precision  in  the  objective  function  value 
with  less  function  evaluations  than  MINOS,  but  MINOS  produces  3-5  place 
precision  with  less  function  calls  than  XS.  MINOS  usually  requires  more 
function  evaluations  than  XS  to  yield  a  feasible  solution. 

For  most  classes  of  functions  (polynomial,  exponential,  etc.),  the 
gradient  is  of  the  same  class  as  the  function;  the  function  calls  (and 
therefore  the  required  CPU  time)  to  determine  gradient  information  are 
approximately  the  same  for  both  analytic  and  numerical  methods. 


A  conversation  late  in  this  research  with  Professor  G.  Vanderplatz, 
Dept.  of  Mechanical  Engineering,  Naval  Postgraduage  School,  revealed 
that  his  widely  used  optimization  codes  for  engineering  design  problems 
have  been  evaluated  using  the  number  of  function  calls  as  the  exclusive 
gauge  of  efficiency,  that  closed-form  gradients  are  rarely  available  for 
such  problems,  and  that  empirical  engineering  data  resolution  may  often 
be  as  poor  as  +_  20  percent. 
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F.   USER  FRIENDLINESS 

Because  of  its  documentation  and  intended  use,  it  was  expected  from 
the  start  of  the  study  that  MINOS  would  be  the  system  most  amenable  to 
the  user  and  the  results  bear  this  out. 

1.  Ease  of  Setup 

This  was  one  area  in  which  MINOS  was  not  necessarily  better. 
The  requirements  for  providing  analytic  gradients  proved  to  be  very  time 
consuming,  notwithstanding  their  aforementioned  use  as  a  trouble-shooting 
aid.   In  addition,  the  need  to  refer  to  a  number  of  documents  for  construc- 
ting the  SPECS  and  MPS  files  (until  "templates"  were  constructed)  was  a 
handicap  to  the  new  user.  Once  the  programming  aids  were  in  place,  construc- 
tion of  a  new  problem  became  quite  straightforward  with  a  minimum  of 
outside  reference  required. 

Although  relatively  little  was  required  of  the  user  for  initial 
setup  on  XS,  becoming  familiar  with  the  techniques  of  using  "pointer" 
variables  required  time;  especially  with  the  dearth  of  documentation. 

Interestingly,  as  the  user  becomes  more  experienced  with  both 
systems,  MINOS  becomes  easier  to  use  with  the  exception  of  the  gradient 
functions,  while  in  some  ways  XS  becomes  more  difficult  as  the  profound 
tuning  capability  of  the  system  becomes  apparent  and  places  more  demands 
on  the  talents  of  the  user/analyst  (i.e.,  appropriate  problem  formulation). 

2.  Debug  Output 

Both  systems  are  capable  of  producing  voluminous  debug  listings 
to  assist  the  user,  and  the  only  limitation  is  the  knowledge  and  patience 
of  the  user  in  their  interpretation. 
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3.  Failure  Mode 

Both  codes  give  comprehensive  output  to  indicate  why  they 
fail.  As  long  as  the  user  is  sophisticated  enough  in  system  use  to 
understand  the  diagnostics,  he  can  usually  intuit  the  cause. 

4.  Robustness 

Default  values  exist  for  all  parameters  in  the  SPECS  file  for 
MINOS  and  experience  has  shown  these  values  to  be  very  robust  with  little 
tuning  required  other  than  specifying  the  problem-specific  size  parameters. 
Although  the  explicit  statement  of  the  objective  function  gradient  is 
recommended  for  MINOS  [Ref  6:  p.  18],  experimentation  with  the  objective 
function  differencing  option  has  revealed  no  significant  change  in 
solution  values  or  CPU  time  between  the  explicit  gradient  and  the  numerical 
differencing  representation. 

Many  of  the  XS  tuning  parameters  are  dependent  on  the  scaling  of 
the  problem,  and  their  robustness  is  in  direct  proportion  with  the  user's 
ability  to  provide  a  well-scaled  problem.  This  just  requires  reasonable 
care  in  the  original  problem  formulation,  but  for  complex  test  problems 
presented  in  completed  form,  reformulation  and  scaling  can  be  vexing. 

G.   SUMMARY 

After  15  months  of  intensive  use  of  both  codes,  it  is  apparent 
that  both  systems  have  achieved  what  their  designers  intended.  MINOS  is 
a  well -documented,  easy-to-use  code  that  reliably  achieves  excellent 
results  on  the  general  (NLP)  problem  while  demanding  only  moderate  skill 
of  the  user.   Its  default  parameters  are  robust  and  require  minimal 
tuning  to  achieve  satisfactory  results.  Although  its  input  files  can  be 
somewhat  cumbersome  to  manage,  they  are  straightforward  and  unlikely  to 
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cause  a  confidence  crisis  for  the  inexperienced  user.  Constraint  gradi- 
ents, on  the  other  hand,  can  be  arduous  to  prepare  and  debug,  even  for 
simple  test  problems.  MINOS  does  not  have  some  of  the  more  sophisticated 
file  editing  and  solution  options,  but  its  developers  candidly  admit  that 
this  was  not  a  design  goal.  MINOS  is  not  capable  of  handling  integer 
variables,  but  again,  was  not  intended  to  do  so.  MINOS  appears  to  be  a 
significant  improvement  over  other  contemporary  codes,  and  a  most  useful 
tool  for  the  modeller/analyst. 

XS,  is  also  exactly  what  it  is  claimed  to  be:  an  experimental 
testbed  for  state-of-the-art  optimization  research.   It  is  very  easy  to 
use,  but  is  not  intended  primarily  for  the  casual,  inexperienced  user. 
Nor  is  XS  specifically  designed  for  nonlinear  models.  XS  provides  many 
flexible  file  editing,  problem  representation,  solution,  and  report 
options,  but  is  designed  for  efficient  custom  applications  to  particular 
classes  of  models.  The  default  nonlinear  feature  of  XS,  operating  with 
one  of  the  problem  representation  "templates,"  provides  a  quick-turnaround 
modelling  enviornment. 

XS  is  designed  to  accommodate  problems  with  no  analytic  gradients, 
poor  data  resolution,  approximated  functions,  and  all  the  attendant 
difficulties  of  real-life  optimization  at  large-scale.  Problems  should 
be  formulated  for  XS  with  realistic  range  intervals  and  meaningful 
penalties  associated  with  constraints.  Accordingly,  XS  can  encounter 
difficulties  with  artificial  test  problems  presented  in  strictest  equality 
form  with  default  (rather  than  modelled)  penalties. 

The  large-scale  nonlinear  integer  capability  of  XS,  combined  with 
basis  factorization  (e.g.,  GUB),  decomposition  facilities,  etc.,  make  it 
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truly  unique  in  the  field  of  large-scale  optimization.   It  is  capable  of 
solving  problems  that  no  other  system  in  the  world  known  to  the  author 
can  attempt.  By  achieving  this,  XS  has  more  than  fulfilled  the  goals  of 
its  developers. 
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APPENDIX  A 
TEST  PROBLEMS  AND  RESULTS 

Problem  1  (Himmelblau  6) 

Source:  [Ref.  30] 

No.  of  variables:  45 

No.  of  constraints:  16  linear  equality  constraints 

Objective  function: 


Minimize:  f(x)  = 


E 


k=l 


E  , 

j=i 


k  I  cJk  +  ln 


nk 

£ 


xjk 


Constraints: 


m*>  =  H  (  H  Eijkxijk 

k=l  \  J-l 


-  b-  =  0   i  =  1.....16 


xjk  2.  °   J  =  1»---»nk   k  =  I|...i7 
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b.'s  and  c-.  's  for  Problem  1 
J  J  k 


1 

0.6529581 

1 

1 

2 

0.281941 

2 

1 

3 

3.705233 

3 

1 

4 

47.00022 

4 

1 

5 

47.02972 

1 

2 

6 

0.08005 

2 

2 

7 

0.08813 

3 

2 

8 

0.04829 

4 

2 

9 

0.0155 

5 

2 

10 

0.0211275 

6 

2 

11 

0.0022725 

7 

2 

12 

0.0 

8 

2 

13 

0.0 

9 

2 

14 

0.0 

10 

2 

15 

0.0 

11 

2 

16 

0.0 

12 

2 

13 

2 

1 

3 

2 

3 

3 

3 

4 

3 

5 

3 

Cjk 

J 

k 

Cjk 

0.0 

6 

3 

0.0 

-7.69 

7 

3 

2.2435 

11.52 

8 

3 

0.0 

36.60 

9 

3 

-39.39 

10.94 

10 

3 

-21.49 

0.0 

11 

3 

-32.84 

0.0 

12 

3 

6.12 

0.0 

13 

3 

0.0 

0.0 

14 

3 

0.0 

0.0 

15 

3 

-1.9028 

0.0 

16 

3 

-2.8889 

2.5966 

17 

3 

-3.3622 

39.39 

18 

3 

-7.4854 

21.35 

1 

4 

-15.639 

32.84 

2 

4 

0.0 

6.26 

3 

4 

21.81 

0.0 

1 

5 

-16.79 

10.45 

2 

5 

0.0 

0.0 

3 

5 

18.9779 

-0.50 

1 

6 

0.0 

0.0 

2 

6 

11.959 

0.0 

1 

7 

0.0 

2 

7 

12.899 
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E-  ..  Data  for  Problem  1 

1  J  K 


\|k 


7       8 


10       11       12       13       14       15 


16 


11 

:21 

:31 

'41 

12 

;22 

:32 

:42 

^52 

l62 

72 

:82 

c92 

10,2 

cll,2 

c12,2 

c13,2 

c13 

c23 

(33 

(43 

(53 

l63 

(73 

(83 

(93 

(10,3 

(11,3 

<12,3 

(13,3 


1     1 


1 

1 

1 

1 

1 

1 

1 

1 

-1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

-1 

1 

1 

-1 

-1 

1 

1 

-1 

-2 

-1 
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kjk 


10       11       12       13       14       15       16 


14,3 

15,3 

16,3 

17,3 

18,3 

14 

:24 

;34 

15 

:25 

:35 

16 

:26 

17 

C27 


-4 


-4 


Alternate  Initial   Points: 


a)     x-k  =  0.01  j=l,...,nk  k=l, ...  ,7 


b)     xjk  =  1.0  j=l,...,nk  k=l, ... ,7 
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Results  for  Problem  1 


Initial 


MINOS 


XS 


f(x) 

-30.958 

-1895.4 

-1910.0 

xll 

0.1 

0.64387 

0.0 

x21 

0.1 

0.0 

0.25803 

x31 

0.1 

3.7052 

3.7052 

x41 

0.1 

0.0 

0.32129 

x12 

0.1 

0.0 

0.64736 

x22 

0.1 

0.0 

0.0 

x32 

0.1 

0.0 

0.0 

x42 

0.1 

0.0 

0.0 

x52 

0.1 

0.72630D- 

■01 

0.0 

x62 

0.1 

0.0 

0.17279D-01 

x72 

0.1 

0.88130D- 

■01 

0.36270D-01 

x82 

0.1 

0.0 

0.0 

x92 

0.1 

46.675 

33.231 

x10,2 

0.1 

0.0 

0.0 

xll,2 

0.1 

0.0 

0.0 

x12,2 

0.1 

0.0 

0.0 

x13,2 

0.1 

0.15500D- 

■01 

0.15500D-01 

x13 

0.1 

0.0 

0.0 

x23 

0.1 

0.0 

0.0 

x33 

0.1 

0.0 

0.0 

x43 

0.1 

0.0 

0.0 

x53 

0.1 

0.0 

0.0 

x63 

0.1 

0.80050D- 

•01 

0.62771D-01 

x73 

0.1 

0.0 

0.51860D-01 

X83 

0.1 

0.48290D- 

■01 

0.48290D-02 

x93 

0.1 

0.30577 

13.476 

x10,3 

0.1 

0.0 

0. 151700-02 

xll,3 

0.1 

0.0 

0.0 

X12,3 

0.1 

0.28194 

0.0 

x13,3 

0.1 

0.21127D- 

-01 

0.21127D-01 

x14,3 

0.1 

0.0 

77 

0.11682D-03 

Initial 


MINOS 


XS 


x15,3 
x16,3 
x17,3 
x18,3 
x14 

O.l 
0.1 
0.1 
0.1 
0.1 

0.0 

0.0 

0.0 

0.22725D- 

0.0 

-02 

0.10080D-02 

0.0 

0.0 

0.11477D-02 

0.0 

x24  ' 

0.1 

0.0 

0.0 

x34 

0.1 

0.90900D- 

-02 

0. 559870-02 

X15 

0.1 

0.0 

0.0 

X25 

0.1 

0.0 

0.0 

X35 

0.1 

0.0 

0.0 

x16 

0.1 

0.3636D-01 

0.0 

x26 

0.1 

0.0 

0.22395D-01 

x17 

0.1 

0.0 

0.0 

x27 

0.1 

0.0 

0.0 

g^x) 

0.647 

0.65296 

0.65296 

92(x) 

0.818 

0.28194 

0.28194 

g3(x) 

-3.405 

3.7052 

3.7052 

g4(x) 

-46.70 

47.000 

47.000 

g5(x) 

-45.93 

47.030 

47.030 

g6(x) 

0.12 

0.80050D- 

-01 

0.80050D-01 

g7(x) 

0.112 

0.88130D- 

■01 

0.88130D-01 

g8(x) 

0.152 

0.48290D- 

-01 

0.48290D-01 

g9(x) 

0.085 

0.15500D- 

-01 

0.15500D-01 

g10(x) 

0.079 

0.21127D- 

-01 

0.21127D-01 

gn(x) 

0.498 

0.22725D- 

-02 

0.22725D-02 

g12(x) 

-1.3 

0.0 

0.0 

g13(x) 

-0.7 

0.0 

0.0 

g14(x) 

0.3 

0.0 

0.0 

g15(x) 

-0.2 

0.0 

0.0 

g16(x) 

-0.2 

0.0 

0.0 
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Problem  2  (Himmelblau  4A) 

Source:  [Ref.  31,  32] 

No.  of  independent  variables:  10 

No.  of  constraints:  3  nonlinear  equality  constraints 

Objective  function: 


Minimize:  f(x)  = 


10  r 


i=l 


10 


Ci  +  x  -  1 


n   E 


i=l 


Constraints: 


g^x)  =e1+2e2+2e3+e6+e10-2=0 

X4      Xfi     Xfi     X7 

g2(x)  =e+2e+e+e-l=0 

x?    x7    xs    xq    xin 
g.(x)  =  e  J  +  e  '   +  e  a  +  2e  y  +  e  iU  -  1  =  0 


where 


cl   =  -6.089    c2  =  -17.164    c3  =  -34.054    c4  =  -5.914 


c5  =  -24.721 


c6  =  -14.986 


c?  =  -24.1000 


c8  =  -10.708 


cg  =  -26.662 


c1Q  =  -22.179 
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Results  for  Problem  2 


Initial 


f(x) 

xl 


*10 
9i(x) 

g2(*) 
g3(x) 


-21.015 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-2.3 

-1.298 

-0.499 

-0.398 


MINOS 

•47.761 

•3.2023 

■1.9124 

•0.24443 

■6.5612 

0.72310 
•7.2742 
•3.5973 
•4.0203 
-3.2884 
•2.3344 

0.0 

0.0 

0.0 


xs 

■48.113 

-3.3543 

-1.9456 

-0.22381 

-7.5599 

-0.72055 

-7.1958 

-3.5789 

-5.1213 

-3.0806 

-2.4085 
0.10383D-0I 
0.21430D-02 
0.15153D-01 


Alternate  Initial   Points 
a)       xi   =  2.0 


b)       xi   =  -5.0 


i=l,...,10 
i=l,. ..,10 
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Problem  3  (Himmelblau  18) 

Source:  [Ref.  33] 

No.  of  variables:  15 

No.  of  constraints:   5  nonlinear  inequality  constraints 

15  bounds  on  independent  variables 
Objective  function: 

10  5  5  5 

Maximize:     f(x)  =        /   A       b^   -     /   t       /  4     yz   -  2       2^    d-;z 


J 
i=l  3=1   i=l  J=l 


where  y  =  ci;jx(10+i) 
and   z=x(1Q+i) 

Constraints: 


2  10 

2  2_j  y   +  3djz2  +  ej  -  zZ       ^jX-jl0    J  -  1. 5 

i=l  i=l 


x-  >_  0   i  =  1, ...  ,15 


81 


Data  for  Prob 

em 

3 

J 

1 

2 

3 

4 

5 

eJ 

-15 

-27 

-36 

-18 

-12 

Clj 

30 

-20 

-10 

32 

-10 

C2j 

-20 

39 

-6 

-31 

32 

C3j 

-10 

-6 

10 

-6 

-10 

C4j 

32 

-31 

-6 

39 

-20 

C5j 

-10 

32 

-10 

-20 

30 

do 

4 

8 

10 

6 

2 

alj 

-16 

2 

0 

1 

0 

a2j 

0 

-2 

0 

0.4 

2 

a3j 

-3.5 

0 

2 

0 

0 

a4j 

0 

-2 

0 

-4 

-1 

d5j 

0 

-9 

-2 

1 

-2.8 

a6j 

2 

0 

-4 

0 

0 

d7j 

-1 

-1 

-1 

-1 

-1 

a8j 

-1 

-2 

-3 

-2 

-1 

a9j 

1 

2 

3 

4 

5 

a10j 

1 

1 

1 

1 

1 

bl        b2 

b3 

b4 

b5           b6 

b7 

b8 

b9 

b10 

-40       -2       - 

.25 

-4 

-4           -1 

-40 

-60 

5 

1 

Alternate  Initial   Points: 


a)  x,   =     5.0  i=l,...,15 

b)  xi  =  15.0  1=1, ...  ,5 
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Results  for  Problem  3 


Initial 


MINOS 


XS 


f(x) 

2400.1 

-32.349 

xl 

1.0D-04 

0.0 

x2 

1.00-04 

0.0 

x3 

1.00-04 

5.1740 

x4 

1.0D-04 

0.0 

x5 

1.00-04 

3.0611 

x6 

1. 00-04 

11.840 

x7 

6.0D-01 

0.0 

x8 

1. 00-04 

0.0 

xg 

1.0D-04 

0.10390 

x10 

1.0D-04 

0.0 

xll 

1.0D-04 

0.30000 

x12 

1. 00-04 

0.33347 

x13 

1.00-04 

0.40000 

x14 

1.0D-04 

0.42831 

x15 

1.0D-04 

0.22396 

g^x) 

4.50-01 

0.0 

g2(x) 

3.3D-01 

0.0 

g3(x) 

2.4D-01 

0.0 

g4(x) 

4.2D-01 

0.0 

g5(x) 

4.80-01 

0.0 

■34.228 

0.0 

0.0 

6.3781 

0.19670 

3.6463 
12.886 

0.0 

0.0 

0.0 

0.0 

0.20177D-01 

0.0 

0.0 

0.36088 

0.26740 

0.51426 

0.14235 

0.0 

0.22845 

0. 405830-01 
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Problem  4  (Himmelblau  16) 

Source:  [Ref.  34] 

No.  of  variables:     9 

No.  of  constraints:  13  nonlinear  inequality  constraints 

1  upper  bound 
Objective  function: 
maximize: 

f(x)  =  0.5(XjX4  -  x2x3  +  x3x9  -  x5xg  +  x5x8  -  XgX7) 


Constraints: 


?        ? 

1  "  x3  *  x4  1  ° 

1  -  4  >  0 

i-xf  -x§>0 

1  *  x[  -  (x2  -  xg)2  >  0 

-  xj2  -   (x2  -  x6)2  >  0 


1       A5 

(x1   -   x7)^   -    (x9   -   XQ)~ 


2       ,..  ,2 


'8y     - 
-      ^;)2  "   <x4  -  x6)2  >  ° 


7 


(x3  -  x7)2  -   (x4  -  x8)2  >  0 
1  -  x2  -   (x8  -  xg)2  >  0 

xlx4  "  X2X3  —  ^ 
x3xg  2.  0 

-x5xg  >  ° 
x5x8  -  x5x7  >  0 

xg  >  0 
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Results  for  Problem  4 


Initial 


MINOS 


XS 


f(x) 

0.0 

-0.86603 

-0.86581 

xl 

1.0 

-0.78028D- 

■01 

0.93171 

x2 

1.0 

-0.67479 

0.36276 

x3 

1.0 

0.82437 

0.16281 

x4 

1.0 

-0.56605 

0.98669 

x5 

1.0 

-0.78028D- 

■01 

0.93573 

v 

6 

1.0 

-0.99695 

0.35193 

x7 

1.0 

0.82437 

0.15144 

x8 

1.0 

-0.24388 

0.98818 

x 
9 

1.0 

0.32216 

0.0 

g^x) 

-1.0 

0.0 

-0.73055D-04 

g2(x) 

0.0 

0.89621 

1.0000 

g3(x) 

-1.0 

0.0 

0.55734D-03 

g4U) 

0.0 

0.0 

0.32201D-03 

g5(x) 

1.0 

0.89621 

0.99987 

g6(x) 

1.0 

0.0 

0.31116D-04 

g7(x) 

1.0 

0.0 

-0.31858D-03 

g8(x) 

1.0 

0.89621 

0.99987 

g9(x) 

0.0 

0.0 

0.57312D-03 

g10(x) 

0.0 

0.60044 

0.86025 

gn(x) 

1.0 

0.26558 

0.0 

g12(x) 

-1.0 

0.25138D- 

-01 

0.0 

g13(x) 

0.0 

0.84089 

0.87137 

Alternative  Initial  Points 


a)    xi  =  1.0 


i=l,...,8    xQ  =  0.0 


b)     xi  =  5.0 


i=l,...,9 
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Problem  5  (Himmelblau  20) 

Source:  [Ref.  35] 
No.  of  variables:  24 

No.  of  constraints:  12  nonlinear  equality  constraints 

2  linear  equality  constraints 
6  nonlinear  inequality  constraints 
24  bounds  on  independent  variables 
Objective  function: 

24 
minimize:   f(x)  =   >    a-x- 

i=l 


Constraints: 

x(i+12) 
a    (y)    - 

CiXi 

9-jVXj    "                         24 

12 

bO+12)    Yj       Xj 

40b  i 

EX 

j  =  13 

j-l 

24 

g13U)  =   JL      Xi 
i=i 

-1  =  0 

=  0  i  =  l... .,12 


86 


12  24 

g14(x)  =  V    -^     +f      S    -T--  l-671-° 

1  =  1  1=13 


'14' 


where  f  =   (0.7302)    (530) 


40 


-    X •     +    X/  • 


9(i+14)(x)  =  j     24(l+12)  +el>°         i  s  1.2,3 

E 

j=i 


xj 


-       X,;,^        +       X 


E 


xo 


xi   >_  0         1   =  1, ...  ,24 
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Data  for  Problem  5 


1 

0.0693 

44.094 

123.7 

31.244 

0.1 

2 

0.0577 

58.12 

31.7 

36.12 

0.3 

3 

0.05 

58.12 

45.7 

34.784 

0.4 

4 

0.20 

137.4 

14.7 

92.7 

0.3 

5 

0.26 

120.9 

84.7 

82.7 

0.6 

6 

0.55 

170.9 

27.7 

91.6 

0.3 

7 

0.06 

62.501 

49.7 

56.708 

8 

0.10 

84.94 

7.1 

82.7 

9 

0.12 

133.425 

2.1 

80.8 

10 

0.18 

82.507 

17.7 

64.517 

11 

0.10 

46.07 

0.85 

49.4 

12 

0.09 

60.097 

0.64 

49.1 

13 

0.0693 

44.094 

14 

0.0577 

58.12 

15 

0.05 

58.12 

16 

0.20 

137.4 

17 

0.26 

120.9 

18 

0.55 

170.9 

19 

0.06 

62.501 

20 

0.10 

84.94 

21 

0.12 

133.425 

22 

0.18 

82.507 

23 

0.10 

46.07 

24 

0.09 

60.097 
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Results  for  Problem  5 


Initial 


MINOS 


XS 


f(x) 

Xi 


*10 

xll 

x12 

x13 

x14 

x15 

X16 

x17 

x18 

x19 

x20 

x21 

x22 

x23 

x24 
9x(x) 

g2(x) 


0.14696 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
0.04 
-2.9D-01 
2.2D-02 


0. 556580-01 
0.0 

0.10725 
0.11139 
0.40506D-14 
0. 115910-13 
0.22064D-14 
0.75541D-01 
0.0 
0.0 

•0.17419D-13 
0.0 

0.11195D-01 
•0.10753D-16 
0.19275 
0.28861 
0.0 
0.0 
0.0 

0.21286 
0.94396D-20 
0.14441D-14 
0.0 

0.45547D-15 
0.40622D-03 
0.0 
0.0 


0.55658D-01 
0.27513D-08 
0.10725 
0.11139 
0.24201D-09 
0.28816D-08 
0,13956D-10 
0.75541D-01 
0. 150940-09 
0.80735D-08 
0.59860D-09 
0.17049D-07 
0.11195D-O1 
0.55220D-07 
0.19275 
0.28861 
0.62272D-09 
0.39282D-07 
0.23459D-10 
0.21286 
0.14615D-09 
0.29088D-08 
0.14346D-08 
0. 380280-08 
0.40622D-03 
0.696660-07 
-0.12910D-07 
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g3(x) 

-1.5D-02 

0.0 

-0.81246D-06 

g4(x) 

2.8D-02 

0.0 

0.26202D-09 

g5(x) 

-5.6D-02 

0.0 

0.17996D-07 

g6(x) 

1.1D-02 

0.0 

0.0 

g7(x) 

-2.4D-02 

0.0 

0.82383D-06 

g8(x) 

5.9D-02 

0.0 

0.85965D-10 

g9(x) 

4.3D-02 

0.0 

0.12482D-08 

g10(x) 

4.1D-02 

0.0 

0.86423D-09 

gn(x) 

1.3D-01 

0.0 

0.55334D-08 

g12(x) 

1.0D-02 

0.0 

-0.14175D-07 

g13(x) 

-4.0D-02 

0.0 

0.0 

g14(x) 

-7.3D-01 

0.0 

0.0 

g15(x) 

1.6D-02 

0.10000 

0.10000 

g16(x) 

2.2D-01 

0.0 

0.64365D-06 

g17(x) 

3.2D-01 

0.0 

0.74807D-06 

g18(x) 

2.2D-01 

0.11601D-01 

0.11600D-01 

g19(x) 

5.2D-01 

0.60000 

0.60000 

g20(x) 

2.2D-01 

0.30000 

0.30000 

Alternative  Initial  Points: 


a)     xi  =  0.08     i  =  l, . . .  ,24 


b)     xi  =  0.02     i=l, — ,24 
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Problem  6  and  Problem  12  (Himmelblau  23) 

Source:  [Ref.  32,  36] 

No.  of  independent  variables:  100 

No.  of  constraints:  12  linear  constraints 

100  lower  bounds  on  the  variables 
Objective  function: 

20       /  5 
minimize:   f(x)  =    /    u.   ,  \    \     a- •     -1 

J-l       \  1=1 


Constraints: 

5. 

7=i 


3 


x^   >  b.      j  =  1,6,10,14,15,16,20 


20 

E 

j  =  l 


Xij    >   -°i        1  =  l--5 


x.  .  >_  0     i  =  1,...,5     j  =  1,...,20 


x- .  e  [integer]  —  Problem  12  only 
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Data  from  Problem  6 


aU 

■s 

Vs 

u.'s 

J 

ij 

1 

2 

3 

4 

5 

1 

1 

.84 

.96 

.92 

30 

60 

2 

.95 

.83 

.95 

.94 

50 

3 

1 

.85 

.96 

.92 

50 

4 

1 

.84 

.96 

.95 

75 

5 

1 

.85 

.96 

.95 

40 

6 

.85 

.81 

.90 

.98 

100 

60 

7 

.90 

.81 

.92 

.98 

35 

8 

.85 

.82 

.91 

1 

30 

9 

.80 

.80 

.92 

1 

25 

10 

.86 

.95 

.96 

.90 

40 

150 

11 

1 

.99 

.91 

.95 

30 

12 

.98 

.98 

.92 

.96 

45 

13 

1 

.99 

.91 

.91 

125 

14 

.88 

.98 

.92 

.98 

50 

200 

15 

.87 

.97 

.98 

.99 

70 

200 

16 

.88 

.98 

.93 

.99 

35 

130 

17 

.85 

.95 

1 

1 

100 

18 

.95 

.84 

.92 

1 

1 

100 

19 

.85 

.93 

1 

1 

100 

20 

.85 

.92 

1 

1 

10 

150 

ci 

200 

100 

300 

150 

250 
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Results  for  Problem  6 


Initial 


MINOS 


XS 


XS  Integer 


xC) 
f(x) 

g2U) 
g3(x) 

94(x) 

g5(x) 
g6(x) 

g7(x) 

g8(x) 

g9(x) 

g10(x) 

gn(x) 

g12(x) 


100.0 

-1755.0 

-1770.0 

-1800.0 

-1660.0 

-1800.0 

-1680.0 

505.0 

410.0 

260.0 

350.0 

130.0 

315.0 

240.0 


---- 

(see  next  page) 

-1735.6 

-1734.5 

50.815 

50.236 

100.00 

100.0 

51.132 

51.133 

58.824 

52.928 

70.000 

70.0 

41.281 

48.139 

62.414 

60.401 

200.00  ■ 

-200.0 

100.00 

-100.0 

300.00 

-300.0 

150.00 

-150.0 

250.00 

-250.0 

-1734.5 

50.0 

100.0 

51.0 

52.0 

70.0 

53.0 

61.0 

-200.0 

-100.0 

-300.0 

-150.0 

-250.0 


Alternative  Initial  Points 


a)     x«j  =  10.0       x2-  =  5.0       x~.  =  15.0 


x4-  =  7.5       x5j  =  12.5       j  =  1.....20 


b)     x..  =10        i  =  1,...,5 
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j  =  1.....20 


Weapon  Type  i 


Target 
J 

1 


Total 


10 


-13.5- 

(33.3) 

40.0 


-1.4- 

(8.5) 

7.0 


5.0 


-23.5- 
(17.7) 
20.0      . 

1.0 

-20.9- 

(13.3) 

12.0 

16.0 

100.0- 

(66.3) 

76.0 

(33.7) 
24.0 

-39.1- 

(38.7) 

38.0 

-27.1- 

(27.0) 

26.0 

-20.3- 

(20.3) 

20.0 

-50.8- 

(50.2) 

50.0 

-50.8- 

(50.2) 

50.0 

-45.4- 
(7.9) 
2.0 

-60.3- 

(49.7) 
54.0) 

-48.6- 

(48.2) 

48.0 

-48.6- 

(48.2) 

48.0 

(15.9) 
11.0 

-23.5- 

(33.6) 

32.0 

(22.6) 
16.0 

-20.9- 

(35.9) 
44.0 

-100.0- 

(100.0) 
100.0 

-39.1- 

(38.7) 

38.0 

-27.1- 

(27.0) 

26.0 

-20.3- 

(20.3) 

20.0 

-51.1- 

(51.1) 

51.0 

-51.1- 

(51.1) 

51.0 

-  denotes  MINOS  variables 
(   )  denotes  XS  variables 
No  brackets  denotes  XS  integer  variables  for  Problem  12. 
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Target 
3 

11 


12 


Total 


13 


14 


15 


16 


17 


18 


19 


20 


-33.2- 

(32.2) 

22.0 

(1.5) 

18.0 

-33.2- 

(33.7) 

40.0 

-40.9- 

(40.5) 

40.0 

-54.0- 

(52.4) 

54.0 

-40.9- 

(40.5) 

40.0 

-54.0- 

(52.4) 

54.0 

(9.6) 
10.0 

-58.8- 

(43.3) 

42.0 

-58.8- 

(52.9) 

42.0 

-26.2- 

(26.5) 

26.0 

-43.8- 

(43.5) 

44.0 

-70.0- 

(70.0) 

44.0 

-24.2- 
(14.1) 
7.0 

-17.0- 

(34.0) 
46.0 

-41.2- 

(48.1) 

46.0 

-3.8- 
(10.1) 
18.0 

-72.0- 

(52.1) 

27.0 

-75.8- 

(62.2) 

27.0 

14.4) 

-57.6- 

(46.0) 

58.0 

-64.2- 

(63.3) 

64.0 

-62.4- 

(60.4) 

61.0 

-57.6- 

(60.4) 

58.0 

-64.2- 

(64.3) 

64.0 

-62.4- 

(60.4) 

61.0 

-200- 

(200) 

200 

-100- 

(100) 
100 

-300- 

(300) 

300 

-150- 

(150) 

150 

-250- 

(250) 

250 

Totals 


-  denotes  MINOS  variables 
(   )  denotes  XS  variables 
No  brackets  denotes  XS  integer  variables  for  Problem  12 
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Problem  7  (Waterman  7) 

Source:  [Ref.  27] 

No.  of  variables:  100 

No.  of  constraints:   1  linear  constraint 

1  nonlinear  constraint 
50  lower  bounds  on  the  variables 
Objective  function: 

50 

EBi(xi  +  50) 
— 


xi 


i=l 


where 


and 


di  =  x(i  +  50)  "  mi 


0 


(x)  = 


_x2 
1     e   /2 


^27 


(J  (r)  =     \(f>   (x 


)dx 
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Constraints: 


50 

E 

1  =  1 


ci  —  +x(i+50)  "mi  <K1 


50 


E  4f  <- 


1=1 


xi  >_  0   i  =  1, ,50 


Kx  =  200,000 
K2  =  300 
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Data  from  Problem  7 


Vs 

Vs 

m-  's 

V 

1 

1000 

i 

100 

100 

2 

1500 

10 

200 

100 

3 

2000 

20 

300 

200 

4 

1100 

17 

200 

100 

5 

1900 

23 

100 

100 

6 

700 

8 

200 

200 

7 

400 

12 

200 

200 

8 

1200 

19 

300 

100 

9 

2000 

2 

500 

200 

10 

1300 

5 

300 

100 

11 

1900 

21 

100 

100 

12 

900 

16 

200 

200 

13 

1400 

13 

400 

200 

14 

1500 

19 

500 

300 

15 

2200 

7 

400 

100 

16 

1700 

4 

300 

100 

17 

1800 

12 

200 

200 

18 

800 

5 

100 

100 

19 

700 

18 

100 

100 

20 

1100 

16 

100 

100 

21 

1000 

14 

200 

100 

22 

1800 

21 

200 

200 

23 

1500 

6 

400 

300 

24 

2100 

6 

500 

100 

25 

1600 

14 

100 

100 
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Li's 

Vs 

m. '  s 

Vs 

26 

700 

2 

100 

100 

27 

2000 

12 

200 

200 

28 

1800 

3 

500 

300 

29 

1700 

1 

200 

200 

30 

700 

18 

300 

200 

31 

1200 

19 

100 

100 

32 

1100 

12 

100 

100 

33 

1700 

9 

500 

100 

34 

600 

8 

300 

100 

35 

400 

1 

200 

100 

36 

1000 

3 

100 

100 

37 

1900 

17 

400 

300 

38 

1500 

15 

200 

200 

39 

1400 

18 

400 

300 

40 

1200 

16 

500 

300 

41 

1300 

5 

100 

100 

42 

1900 

12 

200 

100 

43 

2000 

15 

300 

200 

44 

2200 

20 

400 

200 

45 

800 

23 

100 

100 

46 

1900 

17 

200 

200 

47 

2100 

16 

500 

200 

48 

2000 

4 

500 

300 

49 

500 

8 

100 

100 

50 

900 

12 

100 

100 
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Results  for  Problem  7 

Initial         MINOS  XS 


86.215 
353.95 
183.778 
221.42 
143.77 
151.49 
201.95 
259.13 
224.45 
464.12 
259.13 
157.05 
210.64 
222.45 
464.12 
237.75 
272.13 
259.13 
182.61 
224.16 
142.31 
143.74 
220.12 
464.12 
252.25 
166.27 
246.86 
259.13 


f(x) 

2008.2 

80.744 

xl 

300.0 

440.22 

x2 

300.0 

202.72 

x3 

300.0 

235.98 

x4 

300.0 

153.51 

x5 

300.0 

169.57 

x6 

300.0 

213.66 

x7 

300.0 

180.91 

x8 

300.0 

153.31 

Xg 

300.0 

479.48 

x10 

300.0 

248.16 

xll 

300.0 

174.01 

x12 

300.0 

203.71 

x13 

300.0 

232.75 

x14 

300.0 

286.23 

X15 

300.0 

270.84 

x16 

300.0 

304.74 

x17 

300.0 

253.87 

x18 

300.0 

203.51 

x19 

300.0 

131.70 

x20 

300.0 

156.08 

x21 

300.0 

156.98 

x22 

300.0 

228.41 

x23 

300.0 

336.01 

x24 

300.0 

282.50 

x25 

300.0 

185.38 

x26 

300.0 

276.92 

x27 

300.0 

261.94 

100 


Initial 


MINOS 


XS 


^28 
:29 
:30 
'31 

'32 
'33 
'34 
'35 
:36 
'37 
:38 
'39 
'40 
:41 
'42 
'43 
'44 
'45 
'46 
c47 
;48 
;49 
;50 
(51 
(52 
c53 
(54 
(55 
(56 


300.0 

432.01 

300.0 

500.00 

300.0 

192.82 

300.0 

153.31 

300.0 

169.95 

300.0 

221.19 

300.0 

155.56 

300.0 

290.95 

300.0 

272.94 

300.0 

300.04 

300.0 

230.70 

300.0 

283.96 

300.0 

279.00 

300.0 

248.16 

300.0 

208.39 

300.0 

249.02 

300.0 

241.68 

300.0 

130.00 

300.0 

239.61 

300.0 

249.04 

300.0 

409.47 

300.0 

146.09 

300.0 

158.39 

300.0 

297.80 

300.0 

334.36 

300.0 

554.47 

300.0 

322.41 

300.0 

202.27 

300.0 

543.68 

602.37 

462.028 

187.38 

139.08 

153.28 

211.62 

146.33 

258.89 

242.92 

464.12 

220.40 

464.12 

464.12 

222.23 

186.85 

233.13 

228.57 

118.90 

242.71 

464.12 

602.37 

132.06 

144.48 

304.94 

337.09 

556.60 

323.76 

208.06 

544.60 
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Initial         MINOS  XS 


488.01 
393.27 
887.20 
451.26 
210.34 
480.47 
694.72 
844.03 
543.22 
462.07 
486.77 
269.03 
227.42 
227.08 
332.73 
451.28 
898.59 
646.69 
227.31 
292.90 
486.77 
1051.1 
640.93 
581.52 
221.75 
237.51 
633.51 


x57 

300.0 

523.86 

x58 

300.0 

417.05 

x59 

300.0 

889.14 

x60 

300.0 

456.24 

x61 

300.0 

205.63 

x62 

300.0 

488.59 

x63 

300.0 

695.20 

x64 

300.0 

918.47 

x65 

300.0 

537.44 

x66 

300.0 

457.00 

x67 

300.0 

494.59 

x68 

300.0 

264.64 

x69 

300.0 

227.02 

x70 

300.0 

224.53 

x71 

300.0 

330.59 

x72 

300.0 

452.93 

x73 

300.0 

944.49 

x74 

300.0 

642.53 

X75 

300.0 

222.68 

x76 

300.0 

289.05 

x77 

300.0 

491.82 

x78 

300.0 

1095.5 

x79 

300.0 

636.80 

x80 

300.0 

582.94 

x81 

300.0 

217.05 

x82 

300.0 

234.08 

x83 

300.0 

635.20 
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Initial         MINOS  XS 


x84 

300.0 

456.12 

456.43 

x85 

300.0 

413.00 

416.54 

x86 

300.0 

273.58 

278.46 

x87 

300.0 

827.17 

759.29 

x88 

300.0 

483.20 

483.11 

x89 

300.0 

826.89 

748.54 

x90 

300.0 

945.00 

867.96 

x91 

300.0 

256.24 

261.13 

x92 

300.0 

324.47 

328.27 

x93 

300.0 

576.26 

577.86 

x94 

300.0 

652.20 

653.19 

x95 

300.0 

215.75 

219.24 

x96 

300.0 

468.28 

462.97 

x97 

300.0 

770.32 

704.44 

x98 

300.0 

1068.8 

1016.5 

x99 

300.0 

258.80 

262.13 

X100 

300.0 

237.33 

240.31 

91 

80650.0 

0.34840D+06 

0.2000D+06 

92 

300.0 

300.0 

300.00 

Alternate  Initial  Points: 


a)  xi  =  10    i  =  1, . . . ,100 


b)  x.  =  1000   i  =  1 100 
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Problem  8  (Waterman  8) 

Source:  [Ref.  28] 
No.  of  variables:  46 

No.  of  constraints:   1  nonlinear  inequality  constraint 

1  linear  equality  constraint 
46  lower  bounds  on  the  variables 
Objective  function: 

- 
minimize:   f( 

i=l 


w  S  if  4) 


Constraints 


46 


E"<  .- 


1=1 

I 

1=1 


J]  e^  ♦  ad/  <  S 


where  y-  =  x-  +   >     x. 

J  £  A(i) 


A(i)  consists  of  all  arcs  (in  Figure  7)  that  converge  directly 
and  indirectly  upon  node  i. 

xi  _>  0      i  =  1, ...  ,46 
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Data  from  Problem  8 


a  =  0.05 

T  =  500 

b  =  1.50 

S  =  10000 

cl  = 

5.0 

c2 

= 

4.0 

c3 

=  5.0 

c4  = 

7.0 

c5 

= 

7.0 

c6 

=  8.0 

C7  = 

6.0 

C8 

= 

4.0 

c9 

=  3.0 

C10  = 

12.0 

cll 

= 

14.0 

c12 

=  10.0 

c13  = 

21.0 

c14 

= 

23.0 

c15 

=  8.0 

c16  = 

9.0 

C17 

= 

10.0 

c18 

=  13.0 

c19  = 

20.0 

C20 

= 

5.0 

c21 

=  8.0 

c22  = 

6.0 

c23 

= 

3.0 

c24 

=  8.0 

c25  = 

11.0 

c26 

= 

6.0 

c27 

=  18.0 

c28  = 

15.0 

c29 

= 

10.0 

c30 

=  8.0 

c31  = 

8.0 

c32 

= 

14.0 

c33 

=  11.0 

c34  = 

12.0 

c35 

= 

14.0 

c36 

=  18.0 

c37  = 

16.0 

c38 

= 

9.0 

c39 

=  2.0 

c40  : 

8.0 

c41 

= 

11.0 

c42 

=  12.0 

c43  fc 

11.0 

di  *ci 

c44 
c46 

= 

18.0 
13.0 

i  = 

1.....46 

c45 

=  20.0 
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42 


29 


22 


45 


30 


31 


46 


28 


21 


20 


23 


Fig.  7  Transportation  Routes  for  Problem  8 
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Results  for  Problem  8 


Initial 


MINOS 


XS 


f(x) 

-3.249 

-3.4685 

xl 

40.000 

38.690 

x2 

26.667 

44.648 

x3 

86.667 

36.171 

x4 

10.000 

19.468 

x5 

13.333 

19.357 

x6 

13.333 

17.140 

x7 

13.333 

20.084 

x8 

23.333 

23.498 

x9 

23.333 

31.411 

x10 

3.333 

7.2334 

xll 

3.333 

6.2199 

x12 

3.333 

8.3944 

x13 

3.333 

3.7182 

x14 

3.333 

3.2273 

x15 

3.333 

8.7613 

x16 

3.333 

8.0998 

x17 

3.333 

7.4944 

x18 

6.667 

6.7670 

x19 

3.333 

4.1300 

x20 

46.667 

20.064 

x21 

6.667 

9.9835 

x22 

26.667 

11.205 

x23 

10.000 

15.057 

x24 

10.000 

11.270 

x25 

10.000 

8.9236 

x26 

20.000 

16.456 

x27 

3.333 

1.7413 

x28 

3.333 

3.1113 

Xon 

10.000 

4.8377 

-3.4519 
33.766 
38.106 
57.241 
18.032 
17.223 
15.654 
17.869 
20.499 
28.061 
7.5175 
6.4508 
9.0758 
3.7996 
3.3423 
9.3672 
8.2227 
6.9622 
6.4252 
4.5276 
25.446 
9.2984 
12.408 
13.727 
10.197 
8.4804 
15.948 
2.0711 
3.2189 
4.8102 
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Initial         MINOS  XS 


5.9154 
7.3860 
4.6200 
4.4888 
4.7025 
3.2189 
2.3751 
4.3212 
7.7524 
13.059 
9.0575 
6.2221 
2.0307 
2.0711 
1.4787 
1.4787 
2.0711 
9999.9 
500.0 


x30 

13.333 

5.6615 

x31 

3.333 

7.7517 

x32 

3.333 

4.9097 

x33 

3.333 

4.6872 

x34 

3.333 

4.3489 

x35 

3.333 

3.0116 

x36 

3.333 

2.2564 

x37 

3.333 

4.5998 

x38 

3.333 

7.7970 

x39 

3.333 

13.802 

x40 

3.333 

8.4329 

x41 

3.333 

6.6818 

x42 

3.333 

1.9479 

x43 

3.333 

2.0952 

x44 

3.333 

1.4707 

x45 

3.333 

1.2772 

x46 

3.333 

2.1051 

g^x) 

-10080.33 

10000.0 

g2(x) 

-500.00 

500.00 

Alternate  Initial  Points 


a)     xi  =  10.87     i  =  1, ... ,46 


b)     xi  =  85.0      i  =  1,2,3 


x,  =  10.0      i  =  4,. ..,23 


x.  =  3.26     i  =  24,.. .,46 


108 


Problem  9  (Dean  9— Game) 

Source:  Author 

No.  of  independent  variables:  2 

No.  of  constraints:  1  nonlinear  equality  constraint 

Objective  function: 


maximize:     x^  •  x~ 


Constraints: 


2        2 
xl   ,     x2 


+   ^o   =  1 


(30)2      (23)2 
x,  ,  x„  >_  0 


Results  for  Problem  9 


Initial 

MINOS 

xs 

f(x) 

0.0 

345.00 

344.94 

xl 

0.0 

21.213 

21.091 

x2 

40.0 

16.263 

16.355 

g^x) 

3.0246 

1.0000 

0.99989 
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Problem  10  (Dean--Sortie) 


Source:  [Ref.  37] 
No.  of  independent  variables 
No.  of  constraints: 
Objective  function: 


793 


81  linear  inequality  constraints 


where: 


£ 

Vkj  - 

V 

j=i 

, 

\  1   -  exp 

a.  +  2-J 


i=l 


pid  xu 


Constraints: 


J 

j=l 


xijlV  i   =  !,...,! 


I 


tJ-  l09(1  -  -tJ"  V  -  'i  i  ^    pijxid  ^ 


■if  log  (1  -  Cj)   -  Sj 


j      =     1, . . . , J 
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x,  .  <_  S- ,  for  all  i  and  j 

J 


/     Xi  i  2.  9n^i'     n  =  1>2,...,  no.  of  side  constraints 
J  n 

/     Xii  —  9m^i'     m  =  1»2,...,  no.  of  side  constraints 
JeJm 


1 

1 

31 

0.02 

2 

7 

32 

0.02 

3 

7 

1,6,7,11,13 
16,19,20,27 

0.02 

4 

11 

23,24,25,26 

0.02 

5 

12 

23,24,25,26 

0.02 

m 
1 
2 


2 

8 


m 

30,47,51 
53,60,61 

30,47,51 
53,60,61 


9m 
0.15 

0.25 
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Problem  Data 


ij 


i  =  1 


i  =  2 


i  =  3 


i  =  4 


J  = 
J  = 


1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 


0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.8082 

0.2530 

0.1185 

0.2837 

0.3310 

0.3409 

1.3416 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.2997 

0.2806 


0.1268 

0.0164 

0.0361 

0.0318 

0.0 

0.0616 

0.0369 

0.0406 

0.0530 

0.4104 

0.0132 

0.3559 

0.0236 

0.0507 

0.0 

0.0510 

0.0988 

0.0157 

0.6347 

0.0984 

0.0024 

0.2226 

0.0044 

0.0052 

0.0052 

0.0034 

0.1404 

0.0877 

0.0053 

0.1785 

0.1401 

0.3033 

0.786 

0.0988 

0.5810 

0.0 

0.0 

0.0 

0.0 

0.9123 

0.3888 

0.0 

0.0 

0.1771 

0.3491 


0.3477 

0.0344 

0.2877 

0.2948 

0.4693 

0.0616 

0.1101 

0.0406 

0.0557 

0.6140 

0.0530 

0.3559 

0.0403 

0.3782 

0.1537 

0.1281 

0.1079 

0.0537 

0.6347 

0.1170 

0.0024 

0.7555 

0.0058 

0.0121 

0.0121 

0.0083 

0.2191 

0.0928 

0.0053 

0.1662 

0.1401 

0.3033 

0.1799 

0.1550 

1.3693 

0.2872 

0.0 

0.0 

0.0 

0.9123 

0.3888 

0.0 

0.0 

0.1898 

0.3491 


0.6705 

0.0676 

0.4399 

0.4482 

0.5089 

0.0616 

0.1983 

0.0639 

0.0931 

0.7569 

0.0827 

0.3559 

0.0658 

0.5373 

0.2514 

0.2072 

0.1930 

0.0997 

0.6347 

0.2149 

0.0039 

0.5080 

0.0100 

0.0221 

0.0221 

0.0152 

0.3602 

0.1122 

0.0053 

0.3307 

0.1401 

0.3033 

0.3378 

0.2810 

0.9057 

0.2872 

0.0 

0.0 

0.0 

0.9123 

0.3888 

0.0 

0.0 

0.2126 

0.3491 
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TJ 


i  =  5 


i  =  6 


l  =  7 


J  = 


1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
=  43 
=  44 
=  45 


0.1286 

0.1104 

0.0361 

0.0318 

0.0080 

0.0616 

0.0369 

0.0406 

0.0530 

0.4104 

0.0132 

0.3559 

0.0236 

0.0507 

0.0030 

0.0510 

0.2039 

0.0157 

1.1287 

0.3248 

0.0024 

1.4122 

0.0044 

0.0052 

0.0052 

0.0034 

0.1404 

0.0877 

0.8078 

0.2472 

0.1401 

0.3033 

0.3202 

0.3273 

0.8938 

0.2872 

0.0 

0.0 

0.0 

0.9123 

0.3888 

0.0 

0.0 

0.2177 

0.3491 


0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.6 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.1988 

0.6822 

0.6702 

0.0 

0.0 

0.5679 

0.7527 

0.0 

0.0 


0.1861 

0.1050 

0.0564 

0.0507 

0.0160 

0.0784 

0.0610 

0.0507 

0.0599 

0.4493 

0.0132 

0.4527 

0.0362 

0.0921 

0.0030 

0.0759 

0.1904 

0.0207 

0.8565 

0.3184 

0.0024 

1.3749 

0.0036 

0.0052 

0.0052 

0.0034 

0.1509 

0.1056 

0.8078 

0.2464 

0.1413 

0.2591 

0.3044 

0.3000 

0.7301 

0.4931 

0.0 

0.0 

0.0 

1.0000 

0.6217 

0.0 

0.0 

0.2656 

0.4574 
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1J 


i  =  8 


i  =  9 


i  =  10 


J  = 


1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 


0.1797 

0.0244 

0.0723 

0.0686 

0.0212 

0.0617 

0.0716 

0.0486 

0.0590 

0.4330 

0.0132 

0.4285 

0.0285 

0.1133 

0.0040 

0.0574 

0.0648 

0.0203 

0.6087 

0.1076 

0.0024 

0.3077 

0.0036 

0.0052 

0.0052 

0.0028 

0.1511 

0.1061 

0.0047 

0.1314 

0.0660 

0.2521 

0.0610 

0.0704 

0.6724 

0.2858 

0.0 

0.0 

0.0 

0.8753 

0.4618 

0.0 

0.0 

0.1248 

0.2935 


0.6223 

0.0644 

0.4157 

0.4240 

0.4546 

0.0617 

0.1835 

0.0622 

0.0888 

0.7112 

0.0903 

0.4285 

0.0647 

0.5443 

0.2511 

0.2185 

0.1727 

0.0883 

0.6087 

0.2049 

0.0037 

1.4168 

0.0088 

0.0205 

0.0205 

0.0139 

0.3435 

0.1151 

0.0047 

0.3280 

0.0660 

0.2521 

0.3290 

0.2658 

0.8096 

0.2858 

0.0 

0  0 

0.0 

0.8753 

0.4618 

0.0 

0.0 

0.1763 

0.2935 


0.1818 

0.1036 

0.0727 

0.0690 

0.0222 

0.0629 

0.0733 

0.0511 

0.0613 

0.4467 

0.0132 

0.4385 

0.0296 

0.1160 

0.0100 

0.0585 

0.1795 

0.0209 

0.7022 

0.3107 

0.0024 

1.2910 

0.0036 

0.0052 

0.0052 

0.0028 

0.1570 

0.1099 

0.8096 

0.2557 

0.0660 

0.2521 

0.3067 

0.3091 

0.6919 

0.2953 

0.0 

0.0 

0.0 

0.9032 

0.4761 

0.0 

0.0 

0.2322 

0.2954 
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1J 


1  =  11 


i  =  12 


i  =  13 


1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 


0.5609 

0.1145 

0.3825 

0.3900 

0.4765 

0.1220 

0.1750 

0.0942 

0.1011 

0.7068 

0.0397 

0.4818 

0.0552 

0.5091 

0.2284 

0.1434 

0.1706 

0.0815 

0.7362 

0.3022 

0.0030 

1.3606 

0.0079 

0.0172 

0.0172 

0.0117 

0.3190 

0.2051 

0.8081 

0.2593 

0.1358 

0.3910 

0.2816 

0.1974 

1.0607 

0.3951 

0.0 

0.0 

0.0 

1.0000 

0.5639 

0.0 

0.0 

0.2330 

0.4726 


0.0360 

0.0070 

0.0070 

0.0060 

0.0110 

0.0760 

0.0260 

0.0650 

0.0430 

0.1800 

0.0080 

0.1970 

0.0350 

0.0390 

0.0 

0.0390 

0.1210 

0.0110 

0.4230 

0.1570 

0.0020 

0.0920 

0.0040 

0.0050 

0.0050 

0.0040 

0.0870 

0.0610 

0.0020 

0.0498 

0.1150 

0.1540 

0.0500 

0.0490 

0.0220 

0.2390 

0.0 

0  0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0460 

0.1530 


0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.3483 

0.1053 

0.1782 

0.1782 

0.0 

0.3483 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.1782 
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p. 


TJ 


1     =    1 


i   =  2 


i   =  3 


i   =  4 


j  =  46 

0.3137 

0.1527 

0.1856 

0.3533 

j  =  47 

0.0876 

0.2337 

0.2337 

0.2337 

j  =  48 

0.0819 

0.0593 

0.0471 

0.0932 

j  =  49 

0.0 

0.0 

0.1641 

0.1641 

j  =  50 

0.8139 

0.1934 

0.1934 

0.1934 

j  =  51 

0.2565 

0.4606 

0.1504 

0.2945 

j  =  52 

0.8162 

0.4058 

0.4058 

0.4058 

j  =  53 

0.2569 

0.6677 

0.1804 

0.3529 

j  =  54 

1.3428 

0.5631 

0.7030 

0.7030 

j  =  55 

1.3680 

0.5612 

0.5612 

1.2562 

j  =  56 

0.0 

0.0 

0.0 

0.0 

j  =  57 

0.4781 

0.5573 

0.5573 

0.5573 

j  =  58 

0.0805 

0.0277 

0.0457 

0.0919 

j  =  59 

0.8097 

0.0637 

0.0637 

0.0637 

j  =  60 

0.2510 

0.2293 

0.1421 

0.2830 

j  =  61 

0.0772 

0.1120 

0.1120 

0.1120 

1J 


i   =  5 


l   =  6 


i   =  7 


j  =  46 

0.2707 

0.0 

0.2683 

j  =  47 

0.2337 

0.0 

0.2421 

j  =  48 

0.0819 

0.0 

0.0823 

j  =  49 

0.5366 

0.0 

0.2176 

j  =  50 

0.8148 

0.0 

0.8171 

j  =  51 

0.4076 

0.0 

0.4276 

j  =  52 

0.8176 

0.0 

0.8207 

j  =  53 

0.6316 

0.0 

0.6551 

j  =  54 

1.2897 

0.0 

0.7351 

j  =  55 

1.2234 

0.0 

0.7924 

j  =  56 

0.0 

0.7353 

0.0 

j  =  57 

0.5573 

0.0 

0.6876 

j  =  58 

0.0798 

0.0 

0.0799 

j  =  59 

0.0895 

0.0 

0.8100 

j  =  60 

0.2466 

0.0 

0.2469 

j  =  61 

0.1120 

0.0 

0.1162 
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1J 


i  =  8 


i  =  9 


i  =  10 


j  =  46 

0.1324 

0.3446 

0.2868 

j  =  47 

0.1263 

0.1263 

0.1322 

j  =  48 

0.0284 

0.0920 

0.0844 

j  =  49 

0.1443 

0.1443 

0.4848 

j  =  50 

0.1803 

0.1803 

0.8204 

j  =  51 

0.3479 

0.2922 

0.2678 

j  =  52 

0.3806 

0.3806 

0.8244 

j  =  53 

0.5443 

0.3496 

0.4842 

j  =  54 

0.6039 

1.4349 

0.7827 

j  =  55 

0.5474 

1.3944 

1.0718 

j  =  56 

0.0 

0.0 

0.0 

j  =  57 

0.5145 

0.5145 

0.5287 

j  =  58 

0.0132 

0.0911 

0.0815 

j  =  59 

0.0565 

0.0565 

0.8122 

j  =  60 

0.1643 

0.2804 

0.2553 

j  =  61 

0.0608 

0.0697 

0.0646 

1J 


i  =  11 


i  =  12 


i  =  13 


J  = 


J  = 


46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 


0.2465 

0.2641 

0.0842 

0.4547 

0.8714 

0.5516 

0.8387 

0.7057 

0.8392 

0.6240 

0.0 

0.7425 

0.0827 

0.8134 

0.2914 

0.1245 


0.1620 

0.0640 

0.0344 

0.1790 

0.2243 

0.1415 

0.1518 

0.2870 

0.1900 

0.1460 

0.0 

0.2220 

0.0178 

0.0245 

0.0772 

0.0348 


0.1782 

0.0745 

0.0745 

0.0 

0.7630 

0.2308 

0.7630 

0.2308 

0.0 

0.0 

0.0 

0.3896 

0.0340 

0.3483 

0.1053 

0.0340 
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Problem  Data 


T. 


i  =  l 

500.0000 

i  =  2 

500.0000 

i  =  3 

500.0000 

i  =  4 

500.0000 

i  =  5 

500.0000 

i  =  6 

500.0000 

i  =  7 

500.0000 

i  =  8 

500.0000 

i  =  9 

500.0000 

i  =  10 

300.0000 

)  =  11 

300.0000 

i  =  12 

300.0000 

i  =  13 

300.0000 

j  =  14 

300.0000 

i  =  15 

300.0000 

j  =  16 

300.0000 

i  =  17 

300.0000 

j  =  18 

500.0000 

i  =  19 

500.0000 

j  =  20 

500.0000 

)'  =  21 

500.0000 

i  =  22 

500.0000 

j  =  23 

500.0000 

j  =  24 

900.0000 

j  =  25 

900.0000 

j  =  26 

900.0000 

j  =  27 

900.0000 

j  =  28 

900.0000 

j  =  29 

900.0000 

j  =  30 

900.0000 

j  =  31 

900.0000 

j  =  32 

1500.0000 

j  =  33 

1500.0000 

j  =  34 

500.0000 

j  =  35 

500.0000 

j  =  36 

500.0000 

j  ■  37 

500.0000 

j  =  38 

500.0000 

j  =  39 

700.0000 

j  =  40 

700.0000 

j  =  41 

700.0000 

i  =  42 

700.0000 

j  =  43 

700.0000 

j  =  44 

700.0000 

j  =  45 

400.0000 

^ 

1 . 

c 

J 

J 

J 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.1500 

0.0 

0.0 

0.1500 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1500 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.4500 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.1500 

0.0 

0.0 

0.5500 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.1000 

0.0 

0.0 

0.3000 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.5000 

0.0 

0.0 

0.3000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

vj 


20.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

10.0000 

10.0000 

10.0000 

1.0000 

1.0000 

15.0000 

1.0000 

10.0000 

10.0000 

10.0000 

20.0000 

20.0000 

12.0000 

12.0000 

20.0000 

10.0000 

10.0000 

9.0000 

10.0000 

6.0000 

10.0000 

10.0000 

1.0000 

1.0000 

1.0000 

15.0000 

1.0000 

1.0000 

15.0000 

15.0000 

12.0000 

5.0000 

5.0000 

15.0000 

15.0000 

10.0000 

10.0000 
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VJ 


j  =  46 

400.0000 

j  =  47 

400.0000 

j  =  48 

400.0000 

j  =  49 

400.0000 

j  =  50 

400.0000 

j  =  51 

400.0000 

j  =  52 

5000.0000 

j  =  53 

500.0000 

j  =  54 

500.0000 

j  =  55 

500.0000 

j  =  56 

500.0000 

j  =  57 

900.0000 

j  =  58 

900.0000 

j  =  59 

900.0000 

j  =  60 

300.0000 

j  =  61 

300.0000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.3500 

0.0 

0.0 

0.3000 

0.0 

0.0 

0.4500 

0.0 

0.0 

0.4500 

0.0 

0.0 

0.4500 

0.0 

0.0 

0.2500 

0.0 

0.0 

0.4000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.2000 

0.0 

0.0 

0.4000 

10.0000 

2.0000 

10.0000 

10.0000 

20.0000 

10.0000 

2.0000 

2.0000 

10.0000 

10.0000 

10.0000 

10.0000 

10.0000 

2.0000 

2.0000 

2.0000 
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Results  for  Problem  10:  (XS) 

(only  non-zero  values  of  final  solution  shown) 


f(x) 

-0.20087D+06 

xl,29 

720.33 

x3,54 

570.49 

xl,31 

95.000 

x3,55 

685.66 

xl,44 

2388.6 

x3,57 

21.444 

xl,46 

819.53 

x4,l 

487.50 

xl,50 

11.342 

x4,18 

1708.3 

xl,52 

7.1002 

x4,27 

1414.3 

Xl,54 

33.031 

x4,30 

1009.1 

xl,55 

159.98 

x4,46 

130.84 

xl,58 

106.53 

x5,17 

163.12 

Xl,59 

408.59 

x5,20 

857.98 

x2,19 

359.81 

x5,22 

122.16 

x2,40 

538.20 

x5,29 

125.73 

x2,45 

1215.4 

x5,48 

113.00 

X2,51 

449.35 

x5,49 

574.18 

x2,53 

263.15 

x5,50 

186.12 

x2,55 

87.235 

X5,52 

1814.3 

x2,57 

1836.9 

x5,53 

43.287 

x3,l 

75.840 

x5,54 

169.69 

x3,10 

514.79 

x5,55 

9.5330 

x3,14 

1099.7 

x5,58 

456.53 

x3,19 

111.06 

x5,59 

114.33 

X3,22 

252.46 

x6,38 

935.99 

X3,27 

598.34 

x6,39 

962.27 

x3,35 

466.32 

x6,42 

1084.8 

x3,40 

290.90 

x6,43 

1187.7 

x3,45 

63.036 

x6,56 

579.31 
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x7,20 

x7,22 

x7,29 

x7,32 

x7,41 

x7,44 

x7,48 

X7,51 

x7,52 

x7,53 

x7,59 

X8,9 

X8,28 

x8,40 

x8,41 

x8,51 

X8,53 

x9,l 

x9,ll 

x9,16 

x9,27 

x9,30 

x9,46 

x10,20 

x10,29 

x10,48 

x10,49 

x10,52 

x10,58 

x10,59 

xll,9 

xll,24 

Xll,25 


268.36 

Xll, 

27 

149.05 

129.81 

XH, 

28 

3666.1 

436.42 

XH, 

51 

111.91 

95.000 

x12, 

17 

2491.4 

111.42 

x12, 

19 

932.12 

245.28 

x12, 

20 

607.74 

804.67 

x13, 

23 

95.000 

36.377 

x13, 

46 

623.77 

2316.2 

x13, 

34 

1833.3 

59.708 

x13, 

48 

112.45 

246.82 

x13, 

50 

392.41 

1197.7 

x13, 

52 

2411.9 

748.63 

9l 

4750.0 

56.122 

^2 

4750.0 

1594.3 

^3 

4750.0 

506.04 

94 

4750.0 

647.25 

95 

4750.0 

278.91 

96 

4750.0 

5.3503 

97 

4750.0 

1446.6 

98 

4750.0 

757.37 

99 

4750.0 

2140.1 

910 

4750.0 

121.65 

Sll 

4750.0 

658.93 

912 

4750.0 

41.692 

913 

4750.0 

1120.5 

914 

526.80 

418.16 

922 

144.25 

1120.5 

g23 

316.08 

921.35 

g24 

0.48313 

468.90 

g27 

415.893 

727.91 

929 

316.08 

31.299 

930 

334.72 

63.701 

931 

170.31 
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g32  693.15  g70  1035.7 

g33  664.26  g71  120.10 

g35  541.73  g72  1004.1 

g36  0.38000  g75  712.50 

g37  0.53835  g7fi  1153.3 


693.15 

^70 

664.26 

971 

541.73 

972 

0.38000 

975 

0.53835 

976 

1.0957 

977 

948.24 

978 

831.35 

979 

1070.0 

980 

1035.7 

981 

11.257 

24.614 

638.53 

638.53 

638.53 

644.92 

805.51 

805.51 

616.04 

893.94 

781.00 

446.29 

446.29 

178.42 

510.83 

460.29 

460.29 

6154.0 

594.46 

664.26 

664.26 

425.97 

95.000 
95.000 
268.36 


938 
940 

941 

g42  1070.0  g80  95.000 

g43  1035.7  gR1  95.000 

944 
g45 
g47 
948 
951 
952 
g53 
g54 
g55 
956 
957 
g58 
959 
961 
g62 
g63 
g64 
g65 
g66 
g67 
g68 
g69 
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Problem  11 

See  Section  III  .C.ll 


Problem  12 

See  Problem  6. 


123 


Problem  13  (Dean— Integer  Nonlinear) 

Source:  [Ref.  38] 

No.  of  independent  variables:  12  (8  binary) 

No.  of  constraints:  48  nonlinear  equality  constraint 

3  linear  equality  constraint 

Objective  function: 

I 
minimize:    7     p^(25^n^) 


l x  it 


Constraints: 


i=l 


elik 


T 

_     • 

c 

<  1. 

e2ik 
T 

<   1. 

e2ik 
C 

<  1. 

Y12ik 

+ 

<  1. 

\T 

Y12ik 

<  1, 

i   )    *    «    i    j   i  )        N  1    )    i    •    •    ^  N  ^ 


1  ,  .  •  .  ,  1 ,     K    ■"    1  ,  .  ♦  .  ,  K  , 


1,...,15    K    -    l9...9K9 


~"    l  )  «  •  •  )  i  f     i\    ~*    i)«**)i\) 


1  ,  .  .  .  ,  1 ,     K    -    1 ,  .  .  •  ,  K  , 


*~    i)«**)l)    i\    ~~    i  j  .  •  •  ^  i\  ^ 


IT 
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2  <cii>i  (26ini)>A 


(i) 


i=i 
i 


Yj       (c22>i    (25ini}  i  A 


(1) 

22 


1=1 
I 


£       Oi    (2Vi)>A 


(1) 
66 


1  =  1 


where: 


ni  _>  0,  i   =  1 ,1 

0<   e.   <  90°,  i   =  !,...,! 


e.   =  x.j,  i   =  1.....I 

ni   =  Xj  +  2xj+1  +  4xj+2,     (j  =  3(i   -  1)  +  I) 

2  2 

lik         i   xk         i   yk         ii   xyk 

2  2 

e2ik  =  miexk  +  Vxk  '  mi1iYxyk 

Y12ik  =  "^iVxk  +  ^iVyk  +  (1i   -mi}   Yxyk 


and 

where: 

xk 

= 

Di 

0 

£yk 

= 

D2 
D 

Yxyk 

= 

D3 
0 
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and  where 


D   '  A^lAgjAgg  -  A26)  -  A12(A12A6g  -  A26A16)  +  A16(A12A25  -  A^Ajg) 

Dl  *  Nxk<A22A66  "  A26>  "  Al2(NykA66  "  A26Nxyk>  +  A16< NykA26  "  A22Nxyk> 

°2  '  All<NykA66  -  A26Nxyk>  "  VA12A66  '  A26A16>  +  VA12Nxyk  "  NykA16» 

D3  *  All<A22Nxyk  -  NykA26>  "  A12<A12Nxyk  "  NykA16»  +  Nxk<A12A26  "  A22A16> 
and  where: 


I 


A 


■E 


u    (crs)1(26ini),  r,s  =  1,2,6 

i  =  l 


and  where: 


(Cn),   -(Ctftf  +  2(C12),1^  +  (c22)i^  +  4(C66)nmflf 

(c;2),  -  (cu)/rf  *  (c^d4  +  .*)  ♦  (c^ift  -  «(c66)1i^ 

(C;6)i   "(C^lfo  ♦  (C^J^ra^   -  1^)  -  (C22)im^ 

(C22),   -  (CuJX  +  2(^)^2  ♦  (C22)il*  ♦  4(C66)in.flf 
(C26),   =  (C^l,  ♦  (^1,(1^   -m'l,   -  (C^l? 

(W,  ■  (Cii)i«i'i  -  «cuhmb* +  (CjzJi-i1?  +  (WiCi  -  mi»2 
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and  where 


li 


^ll'i 

(1 

"   \TiVi] 

(c12)i 

VTLiELi 

Ynvn 

(1 

-   \TivTLi^ 

(1 

'   vLTivTLi> 

(c22)1 

ETi 

(1 

"   vLTivTLi> 

<C66>1 

= 

GLTi 

and  where 


LTi   *  LTi 


EL!     ETi 


Problem  data: 


I  =  4;  K  =  3 


a[J^=  0.75D06 

A^)=  0.50D06 

A^=  0.50D06 

«i    =  0.005, 

=  ; 

.,...,4 

PA   -  1. 

=  ; 

-,...,4 

ELi    =  20.0D06, 

=  ; 

L.....4 

ETi    =  1.3D06, 

i   =  ; 

L.....4 

GLTi    =  0.65D06, 

i   =  '. 

L.....4 

vLTi    =  0.304, 

i   =  ; 

L.....4 
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e/,    =  8.25D-03, 

l    - 

4 

e£  =   -5.75D-03, 

i   = 

1           4 

e^    =  6.15D-03, 

i   = 

1.....4 

e^    =  2.306D-02 

i   = 

1           4 

X    ,    .     .     .    ,  T 

+                    "            -9 
'1  Ti      '     T|  Ti    '""    c' 

46D- 

-02 

, , 

1.....4 

xl 

s 

4000.0 

yi 

= 

1000.0 

xyl 

= 

2000.0 

x2 

= 

-3000.0 

9 

y2 

= 

1000.0 

xy2 

= 

-2000.0 

x3 

= 

2000.0 

'y3 

= 

1000.0 

..,-3 

= 

3000.0 
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Results  for 

Problem  13: 

(only  final 

solution  shown) 

f(x) 

0.12000 

nl 

5.4978 

921 

0.45455 

n2 

5.8905 

922 

-0.12123 

n3 

6.7285 

923 

-0.26403 

e4 

7.1540 

924 

-0.26403 

6 
.11 

6.0000 

925 

0.23177 

9 
ft12 

0.0 

926 

-0.33254 

9 

A21 

6.0000 

927 

-0.19951 

9 
a22 

0.0 

928 

0.53207D-01 

9 
a31 

6.0000 

929 

-0.45748 

9 
a32 

0.0 

930 

-0.45748 

9 
«41 

6.0000 

931 

-0.24848 

9 

42 

0.0 

932 

0.35651 

9l 

-0.70122D- 

-02 

g33 

-0.63711 

g2 

0.10061D- 

•01 

934 

0.16992 

93 

0.20118 

935 

-0.45286 

g4 

-0.53655D- 

-01 

g36 

-0.45286 

95 

0.23862 

937 

-0.61069 

96 

0.23862 

g38 

0.87621 

97 

0.25052 

g39 

-0.71607 

98 

-0.35944 

940 

0.19097 

99 

0.55526 

941 

0.24063 

910 

-0.14809 

g42 

0.24063 

9ll 

0.28885 

g43 

-0.20712 

912 

0.28805 

944 

0.29717 

913 

0.60751 

945 

-0.58747 

914 

-0.87164 

g46 

0.15667 

915 

0.56892 

947 

0.48192 

916 

-0.15173 

g48 

0.48192 

917 

-0.78705D- 

-01 

949 

-0.20411 

918 

-0.78705D- 

-01 

950 

0.29286 

g19 

0.44638 

951 

0.42399 

920 

-0.64045 

952 

-0.11308 
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g53  0.67886D-01 

g54  0.67886D-01 

g55  -0.56198D-01 

g56  0.80633D-01 

g57  0.47633 

g58  -0.12703 

g5g  0.21884 

g6Q  0.21884 

g61  0.47443 

g62  -0.68071 

g63  -0.19633D-01 

g64  0. 523600-02 

g65  0. 991560-01 

g66  0.99156D-01 

g67  0.49857 

g68  -0.71534 

g6g  -0.13880 

g70  0.37017D-01 


971 
972 


-0.10797 
-0.10797 


g73  1.1737 

g74  0.51938 

g75  0.50936 
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